[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:13:49 dist-test-slave-39l8 krb5kdc[17859](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:13:49 dist-test-slave-39l8 krb5kdc[17859](info): set up 2 sockets
Aug 07 18:13:49 dist-test-slave-39l8 krb5kdc[17859](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 18:13:51.210220 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.510s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:13:51 dist-test-slave-39l8 krb5kdc[17859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:13:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:13:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.110.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
I20260807 18:13:51.435773 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:44043
--webserver_interface=127.17.110.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:38575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:44043
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:13:51.822639 17877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:51.823216 17877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:51.823441 17877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:51.832137 17877 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:13:51.832290 17877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:51.832393 17877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:51.832494 17877 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:13:51.832602 17877 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:13:51.844738 17877 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:38575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:44043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:44043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:13:51.846346 17877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:51.848062 17877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:51.866619 17882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:51.866771 17883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:51.867090 17877 server_base.cc:1034] running on GCE node
W20260807 18:13:51.867336 17886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:51.868438 17877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:51.870312 17877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:51.871601 17877 hybrid_clock.cc:648] HybridClock initialized: now 1786126431871494 us; error 129 us; skew 500 ppm
I20260807 18:13:51.872099 17877 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:51 dist-test-slave-39l8 krb5kdc[17859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126431, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:51.876451 17877 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:13:51.878695 17877 webserver.cc:460] Webserver started at http://127.17.110.62:44157/ using document root <none> and password file <none>
I20260807 18:13:51.879700 17877 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:51.879875 17877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:51.880226 17877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:51.884110 17877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "6cf7039afa4a4e15b9e6134310418b9b"
format_stamp: "Formatted at 2026-08-07 18:13:51 on dist-test-slave-39l8"
server_key: "95785315c0068fbee1a7e9eda61b5ad8"
server_key_iv: "51145dcfa0d090eefff67c1dc957cc9d"
server_key_version: "encryptionkey@0"
I20260807 18:13:51.885449 17877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "6cf7039afa4a4e15b9e6134310418b9b"
format_stamp: "Formatted at 2026-08-07 18:13:51 on dist-test-slave-39l8"
server_key: "95785315c0068fbee1a7e9eda61b5ad8"
server_key_iv: "51145dcfa0d090eefff67c1dc957cc9d"
server_key_version: "encryptionkey@0"
I20260807 18:13:51.892520 17877 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260807 18:13:51.897718 17894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:51.899706 17877 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 18:13:51.899981 17877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "6cf7039afa4a4e15b9e6134310418b9b"
format_stamp: "Formatted at 2026-08-07 18:13:51 on dist-test-slave-39l8"
server_key: "95785315c0068fbee1a7e9eda61b5ad8"
server_key_iv: "51145dcfa0d090eefff67c1dc957cc9d"
server_key_version: "encryptionkey@0"
I20260807 18:13:51.900321 17877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:51.944180 17877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:51.949805 17877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:51.950131 17877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:51.974941 17877 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:44043
I20260807 18:13:51.974974 17973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:44043 every 8 connection(s)
I20260807 18:13:51.976570 17877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:13:51.979835 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 17877
I20260807 18:13:51.980180 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:13:51.982049 17974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:51.982344 17848 external_mini_cluster.cc:1428] Setting key bf52793fea2ca594cb8dc3c78c3170f2
I20260807 18:13:51.998188 17974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cf7039afa4a4e15b9e6134310418b9b: Bootstrap starting.
I20260807 18:13:52.003995 17974 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6cf7039afa4a4e15b9e6134310418b9b: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:52.006062 17974 log.cc:824] T 00000000000000000000000000000000 P 6cf7039afa4a4e15b9e6134310418b9b: Log is configured to *not* fsync() on all Append() calls
I20260807 18:13:52.010632 17974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cf7039afa4a4e15b9e6134310418b9b: No bootstrap required, opened a new log
I20260807 18:13:52.025655 17974 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6cf7039afa4a4e15b9e6134310418b9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cf7039afa4a4e15b9e6134310418b9b" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 44043 } }
I20260807 18:13:52.026516 17974 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6cf7039afa4a4e15b9e6134310418b9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:52.026695 17974 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6cf7039afa4a4e15b9e6134310418b9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cf7039afa4a4e15b9e6134310418b9b, State: Initialized, Role: FOLLOWER
I20260807 18:13:52.027665 17974 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6cf7039afa4a4e15b9e6134310418b9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cf7039afa4a4e15b9e6134310418b9b" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 44043 } }
I20260807 18:13:52.027949 17974 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6cf7039afa4a4e15b9e6134310418b9b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
Aug 07 18:13:52 dist-test-slave-39l8 krb5kdc[17859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:13:52.028141 17974 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6cf7039afa4a4e15b9e6134310418b9b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:13:52.028415 17974 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6cf7039afa4a4e15b9e6134310418b9b [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:52.032318 17974 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6cf7039afa4a4e15b9e6134310418b9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cf7039afa4a4e15b9e6134310418b9b" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 44043 } }
I20260807 18:13:52.033320 17974 leader_election.cc:302] T 00000000000000000000000000000000 P 6cf7039afa4a4e15b9e6134310418b9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6cf7039afa4a4e15b9e6134310418b9b; no voters:
I20260807 18:13:52.034340 17974 leader_election.cc:288] T 00000000000000000000000000000000 P 6cf7039afa4a4e15b9e6134310418b9b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:13:52.034483 17979 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6cf7039afa4a4e15b9e6134310418b9b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:13:52.036095 17979 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6cf7039afa4a4e15b9e6134310418b9b [term 1 LEADER]: Becoming Leader. State: Replica: 6cf7039afa4a4e15b9e6134310418b9b, State: Running, Role: LEADER
I20260807 18:13:52.036800 17979 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6cf7039afa4a4e15b9e6134310418b9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cf7039afa4a4e15b9e6134310418b9b" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 44043 } }
I20260807 18:13:52.037328 17974 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6cf7039afa4a4e15b9e6134310418b9b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:13:52.042321 17980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cf7039afa4a4e15b9e6134310418b9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6cf7039afa4a4e15b9e6134310418b9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cf7039afa4a4e15b9e6134310418b9b" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 44043 } } }
I20260807 18:13:52.043319 17980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cf7039afa4a4e15b9e6134310418b9b [sys.catalog]: This master's current role is: LEADER
I20260807 18:13:52.043219 17981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cf7039afa4a4e15b9e6134310418b9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6cf7039afa4a4e15b9e6134310418b9b. Latest consensus state: current_term: 1 leader_uuid: "6cf7039afa4a4e15b9e6134310418b9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cf7039afa4a4e15b9e6134310418b9b" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 44043 } } }
I20260807 18:13:52.043565 17981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cf7039afa4a4e15b9e6134310418b9b [sys.catalog]: This master's current role is: LEADER
I20260807 18:13:52.047492 17976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:51.986910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52284 (local address 127.17.110.62:44043)
0807 18:13:51.987886 (+ 976us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:51.987905 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:51.989981 (+ 2076us) server_negotiation.cc:407] Connection header received
0807 18:13:52.003683 (+ 13702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:52.003748 (+ 65us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:52.004523 (+ 775us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:52.005226 (+ 703us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:52.011494 (+ 6268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:52.014788 (+ 3294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:52.018464 (+ 3676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:52.020002 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:52.020333 (+ 331us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:52.029933 (+ 9600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:52.029948 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:52.029974 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:52.030052 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:52.035631 (+ 5579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:52.037123 (+ 1492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:52.037134 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:52.037143 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:52.037550 (+ 407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:52.038399 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:52.038406 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:52.038410 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:52.042369 (+ 3959us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:52.044918 (+ 2549us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:52.045353 (+ 435us) server_negotiation.cc:299] Negotiation successful
0807 18:13:52.045915 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":711,"thread_start_us":407,"threads_started":1}
I20260807 18:13:52.055047 17984 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:13:52.062522 17984 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:13:52.074572 17984 catalog_manager.cc:1347] Generated new cluster ID: 766cbd2fc6f949ecbd8a72198ed8d671
I20260807 18:13:52.074764 17984 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:13:52.110730 17984 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:13:52.112356 17984 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:13:52.128409 17984 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6cf7039afa4a4e15b9e6134310418b9b: Generated new TSK 0
I20260807 18:13:52.129503 17984 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.110.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
I20260807 18:13:52.191865 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.1:0
--local_ip_for_outbound_sockets=127.17.110.1
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:44043
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:38575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:13:52.627112 18005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:52.627475 18005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:52.627592 18005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:52.636881 18005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:52.637053 18005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:52.637265 18005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.1
I20260807 18:13:52.648386 18005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:38575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:44043
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.1
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:13:52.649762 18005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:52.651283 18005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:52.667529 18012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:52.667439 18011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:52.667484 18015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:52.668618 18005 server_base.cc:1034] running on GCE node
I20260807 18:13:52.669392 18005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:52.670595 18005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:52.671880 18005 hybrid_clock.cc:648] HybridClock initialized: now 1786126432671826 us; error 53 us; skew 500 ppm
I20260807 18:13:52.672300 18005 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:52 dist-test-slave-39l8 krb5kdc[17859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126432, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:52.676059 18005 init.cc:377] Logged in from keytab as kudu/127.17.110.1@KRBTEST.COM (short username kudu)
I20260807 18:13:52.678228 18005 webserver.cc:460] Webserver started at http://127.17.110.1:44315/ using document root <none> and password file <none>
I20260807 18:13:52.679090 18005 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:52.679214 18005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:52.679562 18005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:52.683117 18005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-0/data/instance:
uuid: "74ed6cace8ce4a6180369cdb749e7642"
format_stamp: "Formatted at 2026-08-07 18:13:52 on dist-test-slave-39l8"
server_key: "38e6ce788cc0c6c2493b7a2a40f1f401"
server_key_iv: "d97267d0dbfd0686d46ac00290d27656"
server_key_version: "encryptionkey@0"
I20260807 18:13:52.684013 18005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance:
uuid: "74ed6cace8ce4a6180369cdb749e7642"
format_stamp: "Formatted at 2026-08-07 18:13:52 on dist-test-slave-39l8"
server_key: "38e6ce788cc0c6c2493b7a2a40f1f401"
server_key_iv: "d97267d0dbfd0686d46ac00290d27656"
server_key_version: "encryptionkey@0"
I20260807 18:13:52.690261 18005 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 18:13:52.694653 18030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:52.696302 18005 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 18:13:52.696472 18005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-0/wal
uuid: "74ed6cace8ce4a6180369cdb749e7642"
format_stamp: "Formatted at 2026-08-07 18:13:52 on dist-test-slave-39l8"
server_key: "38e6ce788cc0c6c2493b7a2a40f1f401"
server_key_iv: "d97267d0dbfd0686d46ac00290d27656"
server_key_version: "encryptionkey@0"
I20260807 18:13:52.696704 18005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:52.719264 18005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:52.724205 18005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:52.724486 18005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:52.725620 18005 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:13:52.727232 18005 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:13:52.727321 18005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:52.727413 18005 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:13:52.727532 18005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:52.771332 18005 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.1:41723
I20260807 18:13:52.771481 18204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.1:41723 every 8 connection(s)
I20260807 18:13:52.772902 18005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
I20260807 18:13:52.779390 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 18005
I20260807 18:13:52.779659 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance
I20260807 18:13:52.780335 17848 external_mini_cluster.cc:1428] Setting key 12cce452a6eaece8631150006adbde2b
Aug 07 18:13:52 dist-test-slave-39l8 krb5kdc[17859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126432, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:13:52.801520 18207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:52.776031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:43577 (local address 127.17.110.62:44043)
0807 18:13:52.776553 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:52.776558 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:52.777708 (+ 1150us) server_negotiation.cc:407] Connection header received
0807 18:13:52.780244 (+ 2536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:52.780250 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:52.780352 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:52.780480 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:52.782482 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:52.783366 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:52.786818 (+ 3452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:52.787256 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:52.787401 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:52.792260 (+ 4859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:52.792271 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:52.792280 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:52.792338 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:52.796639 (+ 4301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:52.797776 (+ 1137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:52.797786 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:52.797792 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:52.797998 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:52.798893 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:52.798903 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:52.798909 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:52.799375 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:52.799571 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:52.800570 (+ 999us) server_negotiation.cc:299] Negotiation successful
0807 18:13:52.800904 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":285,"threads_started":1}
I20260807 18:13:52.802448 18208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:52.776525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:44043 (local address 127.17.110.1:43577)
0807 18:13:52.777464 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:52.777550 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:52.779892 (+ 2342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:52.780764 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:52.780774 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:52.781460 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:52.782245 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:52.782279 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:52.783593 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:52.783598 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:52.786448 (+ 2850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:52.786491 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:52.787574 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:52.787585 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:52.787965 (+ 380us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:52.788949 (+ 984us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:52.788988 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:52.791861 (+ 2873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:52.796915 (+ 5054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:52.796926 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:52.796944 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:52.797583 (+ 639us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:52.798297 (+ 714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:52.798305 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:52.798311 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:52.798642 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:52.799566 (+ 924us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:52.799574 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:52.800035 (+ 461us) client_negotiation.cc:769] Sending connection context
0807 18:13:52.800391 (+ 356us) client_negotiation.cc:240] Negotiation successful
0807 18:13:52.801006 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"thread_start_us":438,"threads_started":1}
WARNING: no policy specified for kudu/127.17.110.2@KRBTEST.COM; defaulting to no policy
I20260807 18:13:52.804610 18205 heartbeater.cc:347] Connected to a master server at 127.17.110.62:44043
I20260807 18:13:52.805066 18205 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.2@KRBTEST.COM" created.
I20260807 18:13:52.806282 18205 heartbeater.cc:511] Master 127.17.110.62:44043 requested a full tablet report, sending...
I20260807 18:13:52.809439 17920 ts_manager.cc:194] Registered new tserver with Master: 74ed6cace8ce4a6180369cdb749e7642 (127.17.110.1:41723)
I20260807 18:13:52.811322 17920 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.1@KRBTEST.COM'} at 127.17.110.1:43577
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
I20260807 18:13:52.835285 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.2:0
--local_ip_for_outbound_sockets=127.17.110.2
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:44043
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:38575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:13:53.218603 18211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:53.219048 18211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:53.219239 18211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:53.227244 18211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:53.227439 18211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:53.227638 18211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.2
I20260807 18:13:53.238427 18211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:38575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:44043
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.2
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:13:53.240170 18211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:53.241695 18211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:53.257607 18216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:53.257632 18222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:53.257558 18217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:53.258848 18211 server_base.cc:1034] running on GCE node
I20260807 18:13:53.259690 18211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:53.261034 18211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:53.262326 18211 hybrid_clock.cc:648] HybridClock initialized: now 1786126433262245 us; error 83 us; skew 500 ppm
I20260807 18:13:53.262775 18211 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:53 dist-test-slave-39l8 krb5kdc[17859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126433, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:53.267081 18211 init.cc:377] Logged in from keytab as kudu/127.17.110.2@KRBTEST.COM (short username kudu)
I20260807 18:13:53.269191 18211 webserver.cc:460] Webserver started at http://127.17.110.2:41403/ using document root <none> and password file <none>
I20260807 18:13:53.270043 18211 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:53.270291 18211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:53.270685 18211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:53.274065 18211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-1/data/instance:
uuid: "d0eb86882e5046aa8d8afeaaeae9b786"
format_stamp: "Formatted at 2026-08-07 18:13:53 on dist-test-slave-39l8"
server_key: "737d05f3cbdb20f97324734e2f5505b0"
server_key_iv: "b893c7187960cb6a8fe988c234c72896"
server_key_version: "encryptionkey@0"
I20260807 18:13:53.275210 18211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance:
uuid: "d0eb86882e5046aa8d8afeaaeae9b786"
format_stamp: "Formatted at 2026-08-07 18:13:53 on dist-test-slave-39l8"
server_key: "737d05f3cbdb20f97324734e2f5505b0"
server_key_iv: "b893c7187960cb6a8fe988c234c72896"
server_key_version: "encryptionkey@0"
I20260807 18:13:53.281833 18211 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.001s
I20260807 18:13:53.286041 18235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:53.287974 18211 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:13:53.288364 18211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-1/wal
uuid: "d0eb86882e5046aa8d8afeaaeae9b786"
format_stamp: "Formatted at 2026-08-07 18:13:53 on dist-test-slave-39l8"
server_key: "737d05f3cbdb20f97324734e2f5505b0"
server_key_iv: "b893c7187960cb6a8fe988c234c72896"
server_key_version: "encryptionkey@0"
I20260807 18:13:53.288656 18211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:53.330893 18211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:53.336462 18211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:53.336861 18211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:53.338176 18211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:13:53.340042 18211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:13:53.340173 18211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:53.340390 18211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:13:53.340480 18211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:53.385468 18211 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.2:43407
I20260807 18:13:53.385538 18394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.2:43407 every 8 connection(s)
I20260807 18:13:53.387045 18211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
I20260807 18:13:53.388847 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 18211
I20260807 18:13:53.389078 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance
I20260807 18:13:53.389530 17848 external_mini_cluster.cc:1428] Setting key 59572fd9e1f10ad3590e5964057f2f9a
Aug 07 18:13:53 dist-test-slave-39l8 krb5kdc[17859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126433, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.110.3@KRBTEST.COM; defaulting to no policy
I20260807 18:13:53.414536 18398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:53.390264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:55873 (local address 127.17.110.62:44043)
0807 18:13:53.390821 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:53.390828 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:53.392142 (+ 1314us) server_negotiation.cc:407] Connection header received
0807 18:13:53.394463 (+ 2321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:53.394470 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:53.394585 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:53.394717 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:53.397017 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:53.398530 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:53.401089 (+ 2559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:53.402000 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:53.402210 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:53.407137 (+ 4927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:53.407143 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:53.407147 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:53.407189 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:53.410819 (+ 3630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:53.411783 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:53.411789 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:53.411795 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:53.411944 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:53.412482 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:53.412486 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:53.412488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:53.412802 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:53.412976 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:53.413845 (+ 869us) server_negotiation.cc:299] Negotiation successful
0807 18:13:53.414119 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":276,"threads_started":1}
I20260807 18:13:53.415752 18399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:53.390983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:44043 (local address 127.17.110.2:55873)
0807 18:13:53.391903 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:53.391960 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:53.394108 (+ 2148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:53.395075 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:53.395090 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:53.395928 (+ 838us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:53.396750 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:53.396776 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:53.398782 (+ 2006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:53.398789 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:53.400862 (+ 2073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:53.400882 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:53.402346 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:53.402357 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:53.402642 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:53.403543 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:53.403612 (+ 69us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:53.406775 (+ 3163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:53.411046 (+ 4271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:53.411064 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:53.411081 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:53.411612 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:53.412096 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:53.412101 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:53.412105 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:53.412334 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:53.412949 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:53.412959 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:53.413367 (+ 408us) client_negotiation.cc:769] Sending connection context
0807 18:13:53.413732 (+ 365us) client_negotiation.cc:240] Negotiation successful
0807 18:13:53.414418 (+ 686us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":365,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.3@KRBTEST.COM" created.
I20260807 18:13:53.417958 18395 heartbeater.cc:347] Connected to a master server at 127.17.110.62:44043
I20260807 18:13:53.418363 18395 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:53.419343 18395 heartbeater.cc:511] Master 127.17.110.62:44043 requested a full tablet report, sending...
I20260807 18:13:53.421509 17920 ts_manager.cc:194] Registered new tserver with Master: d0eb86882e5046aa8d8afeaaeae9b786 (127.17.110.2:43407)
I20260807 18:13:53.423022 17920 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.2@KRBTEST.COM'} at 127.17.110.2:55873
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
I20260807 18:13:53.446593 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.3:0
--local_ip_for_outbound_sockets=127.17.110.3
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:44043
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:38575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:13:53.815138 18205 heartbeater.cc:503] Master 127.17.110.62:44043 was elected leader, sending a full tablet report...
W20260807 18:13:53.858560 18402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:53.858990 18402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:53.859105 18402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:53.867321 18402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:53.867465 18402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:53.867591 18402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.3
I20260807 18:13:53.879205 18402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:38575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:44043
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.3
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:13:53.880628 18402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:53.882241 18402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:53.899459 18414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:53.899459 18412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:53.899459 18411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:53.901229 18402 server_base.cc:1034] running on GCE node
I20260807 18:13:53.902218 18402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:53.903676 18402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:53.905102 18402 hybrid_clock.cc:648] HybridClock initialized: now 1786126433904976 us; error 104 us; skew 500 ppm
I20260807 18:13:53.905709 18402 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:53 dist-test-slave-39l8 krb5kdc[17859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126433, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:53.909835 18402 init.cc:377] Logged in from keytab as kudu/127.17.110.3@KRBTEST.COM (short username kudu)
I20260807 18:13:53.912026 18402 webserver.cc:460] Webserver started at http://127.17.110.3:38967/ using document root <none> and password file <none>
I20260807 18:13:53.912878 18402 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:53.913038 18402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:53.913429 18402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:53.916971 18402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-2/data/instance:
uuid: "7bd7e2fd0f3b4666bc932922750e36cf"
format_stamp: "Formatted at 2026-08-07 18:13:53 on dist-test-slave-39l8"
server_key: "91ccd3100172db59484cf28c78ae7aec"
server_key_iv: "6defb0b5e664a10b72c93046e0539604"
server_key_version: "encryptionkey@0"
I20260807 18:13:53.917976 18402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance:
uuid: "7bd7e2fd0f3b4666bc932922750e36cf"
format_stamp: "Formatted at 2026-08-07 18:13:53 on dist-test-slave-39l8"
server_key: "91ccd3100172db59484cf28c78ae7aec"
server_key_iv: "6defb0b5e664a10b72c93046e0539604"
server_key_version: "encryptionkey@0"
I20260807 18:13:53.924762 18402 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260807 18:13:53.929286 18422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:53.931177 18402 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 18:13:53.931504 18402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-2/wal
uuid: "7bd7e2fd0f3b4666bc932922750e36cf"
format_stamp: "Formatted at 2026-08-07 18:13:53 on dist-test-slave-39l8"
server_key: "91ccd3100172db59484cf28c78ae7aec"
server_key_iv: "6defb0b5e664a10b72c93046e0539604"
server_key_version: "encryptionkey@0"
I20260807 18:13:53.931835 18402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:53.963368 18402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:53.968551 18402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:53.968914 18402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:53.970091 18402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:13:53.971854 18402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:13:53.971937 18402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:53.972102 18402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:13:53.972159 18402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:54.024434 18402 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.3:35969
I20260807 18:13:54.024487 18586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.3:35969 every 8 connection(s)
I20260807 18:13:54.026206 18402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
I20260807 18:13:54.034200 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 18402
I20260807 18:13:54.034451 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance
I20260807 18:13:54.035030 17848 external_mini_cluster.cc:1428] Setting key bbe6f93a2b58f1736266d8a6528450c6
Aug 07 18:13:54 dist-test-slave-39l8 krb5kdc[17859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126433, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:13:54.052546 18589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:54.029562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:39535 (local address 127.17.110.62:44043)
0807 18:13:54.030061 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:54.030066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:54.031437 (+ 1371us) server_negotiation.cc:407] Connection header received
0807 18:13:54.033804 (+ 2367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:54.033810 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:54.033919 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:54.034065 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:54.036227 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:54.037357 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:54.039798 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:54.040279 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:54.040394 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:54.044732 (+ 4338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:54.044739 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:54.044743 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:54.044787 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:54.047962 (+ 3175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:54.049089 (+ 1127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:54.049096 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:54.049098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:54.049276 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:54.050020 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:54.050025 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:54.050027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:54.050356 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:54.050498 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:54.051774 (+ 1276us) server_negotiation.cc:299] Negotiation successful
0807 18:13:54.052079 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":244,"threads_started":1}
I20260807 18:13:54.053907 18590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:54.030123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:44043 (local address 127.17.110.3:39535)
0807 18:13:54.031231 (+ 1108us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:54.031291 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:54.033400 (+ 2109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:54.034389 (+ 989us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:54.034405 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:54.035202 (+ 797us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:54.035915 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:54.035938 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:54.037572 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:54.037580 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:54.039573 (+ 1993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:54.039595 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:54.040527 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:54.040534 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:54.040775 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:54.041549 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:54.041575 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:54.044346 (+ 2771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:54.048209 (+ 3863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:54.048220 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:54.048287 (+ 67us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:54.048869 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:54.049486 (+ 617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:54.049491 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:54.049493 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:54.049835 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:54.050630 (+ 795us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:54.050642 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:54.051232 (+ 590us) client_negotiation.cc:769] Sending connection context
0807 18:13:54.051606 (+ 374us) client_negotiation.cc:240] Negotiation successful
0807 18:13:54.052319 (+ 713us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":830,"thread_start_us":372,"threads_started":1}
I20260807 18:13:54.055968 18587 heartbeater.cc:347] Connected to a master server at 127.17.110.62:44043
I20260807 18:13:54.056345 18587 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:54.057201 18587 heartbeater.cc:511] Master 127.17.110.62:44043 requested a full tablet report, sending...
I20260807 18:13:54.059165 17920 ts_manager.cc:194] Registered new tserver with Master: 7bd7e2fd0f3b4666bc932922750e36cf (127.17.110.3:35969)
I20260807 18:13:54.060045 17920 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.3@KRBTEST.COM'} at 127.17.110.3:39535
I20260807 18:13:54.067025 17848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:13:54 dist-test-slave-39l8 krb5kdc[17859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126434, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 18:13:54 dist-test-slave-39l8 krb5kdc[17859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126434, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.110.1@KRBTEST.COM
I20260807 18:13:54.132484 18596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:54.112503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39566 (local address 127.17.110.1:41723)
0807 18:13:54.113528 (+ 1025us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:54.113535 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:54.113566 (+ 31us) server_negotiation.cc:407] Connection header received
0807 18:13:54.113633 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:54.113639 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:54.113909 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:54.114074 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:54.115823 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:54.116999 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:54.118696 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:54.119674 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:54.119806 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:54.125034 (+ 5228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:54.125052 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:54.125071 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:54.125166 (+ 95us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:54.128783 (+ 3617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:54.129839 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:54.129849 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:54.129852 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:54.130035 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:54.130635 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:54.130640 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:54.130642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:54.131213 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:54.131500 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:54.131889 (+ 389us) server_negotiation.cc:299] Negotiation successful
0807 18:13:54.132167 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":844,"thread_start_us":665,"threads_started":1}
W20260807 18:13:54.134315 18066 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39566
Aug 07 18:13:54 dist-test-slave-39l8 krb5kdc[17859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 18:13:54 dist-test-slave-39l8 krb5kdc[17859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.1@KRBTEST.COM
I20260807 18:13:54.196331 18596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:54.179117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39570 (local address 127.17.110.1:41723)
0807 18:13:54.179393 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:54.179403 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:54.179851 (+ 448us) server_negotiation.cc:407] Connection header received
0807 18:13:54.180110 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:54.180114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:54.180223 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:54.180342 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:54.182065 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:54.182860 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:54.184319 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:54.184771 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:54.184923 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:54.189444 (+ 4521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:54.189450 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:54.189454 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:54.189500 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:54.192720 (+ 3220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:54.193761 (+ 1041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:54.193776 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:54.193778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:54.193950 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:54.194450 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:54.194458 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:54.194467 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:54.194900 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:54.195263 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:54.195644 (+ 381us) server_negotiation.cc:299] Negotiation successful
0807 18:13:54.195930 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 18:13:54.198238 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 18005
I20260807 18:13:54.213935 18596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:54.199411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39576 (local address 127.17.110.1:41723)
0807 18:13:54.199602 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:54.199606 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:54.199959 (+ 353us) server_negotiation.cc:407] Connection header received
0807 18:13:54.200097 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:54.200134 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:54.200236 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:54.200342 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:54.201758 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:54.202502 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:54.204106 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:54.204652 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:54.204764 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:54.207765 (+ 3001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:54.207770 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:54.207773 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:54.207814 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:54.210954 (+ 3140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:54.211790 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:54.211795 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:54.211797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:54.211961 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:54.212462 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:54.212467 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:54.212469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:54.212821 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:54.213103 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:54.213430 (+ 327us) server_negotiation.cc:299] Negotiation successful
0807 18:13:54.213636 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 18:13:54.214907 18200 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:54.304456 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 18005
I20260807 18:13:54.335347 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 18211
Aug 07 18:13:54 dist-test-slave-39l8 krb5kdc[17859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.2@KRBTEST.COM
I20260807 18:13:54.355418 18604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:54.336446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52590 (local address 127.17.110.2:43407)
0807 18:13:54.337039 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:54.337047 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:54.337103 (+ 56us) server_negotiation.cc:407] Connection header received
0807 18:13:54.337211 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:54.337218 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:54.337446 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:54.337592 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:54.339139 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:54.340304 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:54.341730 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:54.342460 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:54.342598 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:54.347693 (+ 5095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:54.347715 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:54.347733 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:54.347795 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:54.351378 (+ 3583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:54.352380 (+ 1002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:54.352390 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:54.352393 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:54.352624 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:54.353289 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:54.353294 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:54.353296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:54.353767 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:54.354116 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:54.354772 (+ 656us) server_negotiation.cc:299] Negotiation successful
0807 18:13:54.355083 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":299,"threads_started":1}
I20260807 18:13:54.357115 18389 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:54.445084 18395 heartbeater.cc:503] Master 127.17.110.62:44043 was elected leader, sending a full tablet report...
I20260807 18:13:54.446442 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 18211
W20260807 18:13:54.475807 17898 connection.cc:535] server connection from 127.17.110.2:55873 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 18:13:54.475948 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 18402
Aug 07 18:13:54 dist-test-slave-39l8 krb5kdc[17859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.3@KRBTEST.COM
I20260807 18:13:54.495226 18607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:54.477147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41716 (local address 127.17.110.3:35969)
0807 18:13:54.477813 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:54.477821 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:54.477854 (+ 33us) server_negotiation.cc:407] Connection header received
0807 18:13:54.477922 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:54.477928 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:54.478156 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:54.478300 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:54.479827 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:54.480894 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:54.482382 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:54.483214 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:54.483378 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:54.488350 (+ 4972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:54.488366 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:54.488384 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:54.488430 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:54.491567 (+ 3137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:54.492470 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:54.492480 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:54.492482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:54.492646 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:54.493243 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:54.493249 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:54.493251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:54.493728 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:54.494102 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:54.494378 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 18:13:54.494633 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":270,"threads_started":1}
I20260807 18:13:54.496788 18581 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:54.589915 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 18402
I20260807 18:13:54.616840 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 17877
I20260807 18:13:54.618218 17967 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:54.725653 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 17877
2026-08-07T18:13:54Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (5098 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:13:54 dist-test-slave-39l8 krb5kdc[18612](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:13:54 dist-test-slave-39l8 krb5kdc[18612](info): set up 2 sockets
Aug 07 18:13:54 dist-test-slave-39l8 krb5kdc[18612](info): commencing operation
krb5kdc: starting...
W20260807 18:13:56.082212 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.301s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:13:56 dist-test-slave-39l8 krb5kdc[18612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:13:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:13:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.110.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
I20260807 18:13:56.332312 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:34185
--webserver_interface=127.17.110.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:42023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:34185
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:13:56.713987 18632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:56.714344 18632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:56.714519 18632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:56.722285 18632 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:13:56.722425 18632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:56.722474 18632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:56.722595 18632 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:13:56.722689 18632 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:13:56.733224 18632 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:42023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:34185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:34185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:13:56.734758 18632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:56.736282 18632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:56.749133 18643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:56.749178 18640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:56.749411 18645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:56.750119 18632 server_base.cc:1034] running on GCE node
I20260807 18:13:56.751127 18632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:56.752700 18632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:56.753957 18632 hybrid_clock.cc:648] HybridClock initialized: now 1786126436753909 us; error 46 us; skew 500 ppm
I20260807 18:13:56.754370 18632 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:56 dist-test-slave-39l8 krb5kdc[18612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126436, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:56.757818 18632 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:13:56.759528 18632 webserver.cc:460] Webserver started at http://127.17.110.62:40867/ using document root <none> and password file <none>
I20260807 18:13:56.760278 18632 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:56.760389 18632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:56.760681 18632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:56.764611 18632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "9f1ca87ef6c948f58f13ec26c1f0eb76"
format_stamp: "Formatted at 2026-08-07 18:13:56 on dist-test-slave-39l8"
server_key: "6141c89eed4f903c4d7da49cc0f3e0d5"
server_key_iv: "0277ce42d925bd755e9d9a2289d36c44"
server_key_version: "encryptionkey@0"
I20260807 18:13:56.765681 18632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "9f1ca87ef6c948f58f13ec26c1f0eb76"
format_stamp: "Formatted at 2026-08-07 18:13:56 on dist-test-slave-39l8"
server_key: "6141c89eed4f903c4d7da49cc0f3e0d5"
server_key_iv: "0277ce42d925bd755e9d9a2289d36c44"
server_key_version: "encryptionkey@0"
I20260807 18:13:56.772408 18632 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 18:13:56.777245 18652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:56.779237 18632 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260807 18:13:56.779448 18632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "9f1ca87ef6c948f58f13ec26c1f0eb76"
format_stamp: "Formatted at 2026-08-07 18:13:56 on dist-test-slave-39l8"
server_key: "6141c89eed4f903c4d7da49cc0f3e0d5"
server_key_iv: "0277ce42d925bd755e9d9a2289d36c44"
server_key_version: "encryptionkey@0"
I20260807 18:13:56.779842 18632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:56.803860 18632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:56.810359 18632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:56.810650 18632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:56.833122 18632 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:34185
I20260807 18:13:56.833177 18724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:34185 every 8 connection(s)
I20260807 18:13:56.834749 18632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:13:56.837757 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 18632
I20260807 18:13:56.838045 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:13:56.838989 17848 external_mini_cluster.cc:1428] Setting key 4b6be2b4c765ba1667578eb6ead9caff
I20260807 18:13:56.840065 18725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 18:13:56 dist-test-slave-39l8 krb5kdc[18612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:13:56.856985 18725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f1ca87ef6c948f58f13ec26c1f0eb76: Bootstrap starting.
I20260807 18:13:56.861413 18725 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f1ca87ef6c948f58f13ec26c1f0eb76: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:56.862641 18725 log.cc:824] T 00000000000000000000000000000000 P 9f1ca87ef6c948f58f13ec26c1f0eb76: Log is configured to *not* fsync() on all Append() calls
I20260807 18:13:56.867198 18725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f1ca87ef6c948f58f13ec26c1f0eb76: No bootstrap required, opened a new log
I20260807 18:13:56.867282 18729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:56.841444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48656 (local address 127.17.110.62:34185)
0807 18:13:56.842371 (+ 927us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:56.842384 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:56.842440 (+ 56us) server_negotiation.cc:407] Connection header received
0807 18:13:56.844184 (+ 1744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:56.844210 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:56.844697 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:56.845190 (+ 493us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:56.846951 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:56.848703 (+ 1752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:56.850692 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:56.851918 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:56.852240 (+ 322us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:56.857330 (+ 5090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:56.857348 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:56.857375 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:56.857438 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:56.861153 (+ 3715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:56.862206 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:56.862215 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:56.862223 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:56.862443 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:56.863050 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:56.863057 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:56.863061 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:56.864042 (+ 981us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:56.864323 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:56.865264 (+ 941us) server_negotiation.cc:299] Negotiation successful
0807 18:13:56.865777 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":428,"threads_started":1}
I20260807 18:13:56.877202 18725 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f1ca87ef6c948f58f13ec26c1f0eb76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f1ca87ef6c948f58f13ec26c1f0eb76" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 34185 } }
I20260807 18:13:56.877830 18725 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f1ca87ef6c948f58f13ec26c1f0eb76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:56.877966 18725 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f1ca87ef6c948f58f13ec26c1f0eb76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1ca87ef6c948f58f13ec26c1f0eb76, State: Initialized, Role: FOLLOWER
I20260807 18:13:56.878628 18725 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f1ca87ef6c948f58f13ec26c1f0eb76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f1ca87ef6c948f58f13ec26c1f0eb76" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 34185 } }
I20260807 18:13:56.879020 18725 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f1ca87ef6c948f58f13ec26c1f0eb76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:13:56.879142 18725 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f1ca87ef6c948f58f13ec26c1f0eb76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:13:56.879300 18725 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f1ca87ef6c948f58f13ec26c1f0eb76 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:56.882179 18725 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f1ca87ef6c948f58f13ec26c1f0eb76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f1ca87ef6c948f58f13ec26c1f0eb76" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 34185 } }
I20260807 18:13:56.882956 18725 leader_election.cc:302] T 00000000000000000000000000000000 P 9f1ca87ef6c948f58f13ec26c1f0eb76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f1ca87ef6c948f58f13ec26c1f0eb76; no voters:
I20260807 18:13:56.883688 18725 leader_election.cc:288] T 00000000000000000000000000000000 P 9f1ca87ef6c948f58f13ec26c1f0eb76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:13:56.883980 18732 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f1ca87ef6c948f58f13ec26c1f0eb76 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:13:56.884853 18732 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f1ca87ef6c948f58f13ec26c1f0eb76 [term 1 LEADER]: Becoming Leader. State: Replica: 9f1ca87ef6c948f58f13ec26c1f0eb76, State: Running, Role: LEADER
I20260807 18:13:56.885669 18732 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f1ca87ef6c948f58f13ec26c1f0eb76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f1ca87ef6c948f58f13ec26c1f0eb76" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 34185 } }
I20260807 18:13:56.886662 18725 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f1ca87ef6c948f58f13ec26c1f0eb76 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:13:56.889499 18733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f1ca87ef6c948f58f13ec26c1f0eb76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f1ca87ef6c948f58f13ec26c1f0eb76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f1ca87ef6c948f58f13ec26c1f0eb76" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 34185 } } }
I20260807 18:13:56.889580 18739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f1ca87ef6c948f58f13ec26c1f0eb76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f1ca87ef6c948f58f13ec26c1f0eb76. Latest consensus state: current_term: 1 leader_uuid: "9f1ca87ef6c948f58f13ec26c1f0eb76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f1ca87ef6c948f58f13ec26c1f0eb76" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 34185 } } }
I20260807 18:13:56.889971 18733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f1ca87ef6c948f58f13ec26c1f0eb76 [sys.catalog]: This master's current role is: LEADER
I20260807 18:13:56.890013 18739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f1ca87ef6c948f58f13ec26c1f0eb76 [sys.catalog]: This master's current role is: LEADER
I20260807 18:13:56.891113 18744 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:13:56.898456 18744 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:13:56.911612 18744 catalog_manager.cc:1347] Generated new cluster ID: c9c232f324ac48918fc9d6a1484d8def
I20260807 18:13:56.911789 18744 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:13:56.931250 18744 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:13:56.933027 18744 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:13:56.945240 18744 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f1ca87ef6c948f58f13ec26c1f0eb76: Generated new TSK 0
I20260807 18:13:56.946420 18744 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.110.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
I20260807 18:13:57.019847 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.1:0
--local_ip_for_outbound_sockets=127.17.110.1
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:34185
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:42023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:13:57.423738 18763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:57.424104 18763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:57.424219 18763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:57.432574 18763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:57.432717 18763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:57.432880 18763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.1
I20260807 18:13:57.443830 18763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:42023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:34185
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.1
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:13:57.445540 18763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:57.447261 18763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:57.463027 18772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:57.462996 18770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:57.463338 18769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:57.464030 18763 server_base.cc:1034] running on GCE node
I20260807 18:13:57.464907 18763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:57.466070 18763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:57.467321 18763 hybrid_clock.cc:648] HybridClock initialized: now 1786126437467264 us; error 49 us; skew 500 ppm
I20260807 18:13:57.467695 18763 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:57 dist-test-slave-39l8 krb5kdc[18612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126437, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:57.471376 18763 init.cc:377] Logged in from keytab as kudu/127.17.110.1@KRBTEST.COM (short username kudu)
I20260807 18:13:57.473277 18763 webserver.cc:460] Webserver started at http://127.17.110.1:34419/ using document root <none> and password file <none>
I20260807 18:13:57.474135 18763 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:57.474313 18763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:57.474742 18763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:57.478266 18763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-0/data/instance:
uuid: "c582a99a91b34101b4a741614ea2d531"
format_stamp: "Formatted at 2026-08-07 18:13:57 on dist-test-slave-39l8"
server_key: "98bae53837492256d7941efeea981ec2"
server_key_iv: "ed689e61d71a6e0dd88d9d5fd4030e54"
server_key_version: "encryptionkey@0"
I20260807 18:13:57.479432 18763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance:
uuid: "c582a99a91b34101b4a741614ea2d531"
format_stamp: "Formatted at 2026-08-07 18:13:57 on dist-test-slave-39l8"
server_key: "98bae53837492256d7941efeea981ec2"
server_key_iv: "ed689e61d71a6e0dd88d9d5fd4030e54"
server_key_version: "encryptionkey@0"
I20260807 18:13:57.486536 18763 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260807 18:13:57.491571 18781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:57.493497 18763 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:13:57.493734 18763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-0/wal
uuid: "c582a99a91b34101b4a741614ea2d531"
format_stamp: "Formatted at 2026-08-07 18:13:57 on dist-test-slave-39l8"
server_key: "98bae53837492256d7941efeea981ec2"
server_key_iv: "ed689e61d71a6e0dd88d9d5fd4030e54"
server_key_version: "encryptionkey@0"
I20260807 18:13:57.493947 18763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:57.514477 18763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:57.520166 18763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:57.520471 18763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:57.521497 18763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:13:57.523139 18763 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:13:57.523231 18763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:57.523334 18763 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:13:57.523382 18763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:57.563333 18763 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.1:38049
I20260807 18:13:57.563458 18944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.1:38049 every 8 connection(s)
I20260807 18:13:57.564881 18763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
I20260807 18:13:57.575330 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 18763
I20260807 18:13:57.575582 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance
I20260807 18:13:57.576460 17848 external_mini_cluster.cc:1428] Setting key b290cf121d63087cfdbe34d4c0b234e8
Aug 07 18:13:57 dist-test-slave-39l8 krb5kdc[18612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126437, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:13:57.591823 18948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:57.568410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:52759 (local address 127.17.110.62:34185)
0807 18:13:57.569194 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:57.569199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:57.570431 (+ 1232us) server_negotiation.cc:407] Connection header received
0807 18:13:57.572548 (+ 2117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:57.572554 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:57.572666 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:57.572794 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:57.574504 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:57.575379 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:57.578007 (+ 2628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:57.578615 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:57.578858 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:57.584014 (+ 5156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:57.584022 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:57.584030 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:57.584083 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:57.587546 (+ 3463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:57.588739 (+ 1193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:57.588747 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:57.588752 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:57.588903 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:57.589518 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:57.589524 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:57.589528 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:57.589860 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:57.590037 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:57.591153 (+ 1116us) server_negotiation.cc:299] Negotiation successful
0807 18:13:57.591429 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":521,"threads_started":1}
I20260807 18:13:57.593079 18949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:57.569273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:34185 (local address 127.17.110.1:52759)
0807 18:13:57.570227 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:57.570271 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:57.572214 (+ 1943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:57.573083 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:57.573092 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:57.573686 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:57.574302 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:57.574322 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:57.575649 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:57.575655 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:57.577736 (+ 2081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:57.577755 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:57.579015 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:57.579025 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:57.579414 (+ 389us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:57.580462 (+ 1048us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:57.580509 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:57.583691 (+ 3182us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:57.587782 (+ 4091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:57.587793 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:57.587810 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:57.588575 (+ 765us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:57.589067 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:57.589072 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:57.589074 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:57.589368 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:57.590029 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:57.590038 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:57.590505 (+ 467us) client_negotiation.cc:769] Sending connection context
0807 18:13:57.590983 (+ 478us) client_negotiation.cc:240] Negotiation successful
0807 18:13:57.591637 (+ 654us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"thread_start_us":445,"threads_started":1}
I20260807 18:13:57.595221 18945 heartbeater.cc:347] Connected to a master server at 127.17.110.62:34185
I20260807 18:13:57.595573 18945 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:57.596434 18945 heartbeater.cc:511] Master 127.17.110.62:34185 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.17.110.2@KRBTEST.COM; defaulting to no policy
I20260807 18:13:57.599233 18673 ts_manager.cc:194] Registered new tserver with Master: c582a99a91b34101b4a741614ea2d531 (127.17.110.1:38049)
I20260807 18:13:57.601002 18673 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.1@KRBTEST.COM'} at 127.17.110.1:52759
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
I20260807 18:13:57.631599 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.2:0
--local_ip_for_outbound_sockets=127.17.110.2
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:34185
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:42023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:13:58.029284 18952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:58.029635 18952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:58.029799 18952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:58.037606 18952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:58.037750 18952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:58.037930 18952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.2
I20260807 18:13:58.048382 18952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:42023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:34185
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.2
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:13:58.049813 18952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:58.051358 18952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:58.065815 18960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:58.065819 18964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:58.065770 18961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:58.067117 18952 server_base.cc:1034] running on GCE node
I20260807 18:13:58.067889 18952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:58.068933 18952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:58.070233 18952 hybrid_clock.cc:648] HybridClock initialized: now 1786126438070194 us; error 46 us; skew 500 ppm
I20260807 18:13:58.070632 18952 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:58 dist-test-slave-39l8 krb5kdc[18612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126438, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:58.074208 18952 init.cc:377] Logged in from keytab as kudu/127.17.110.2@KRBTEST.COM (short username kudu)
I20260807 18:13:58.076121 18952 webserver.cc:460] Webserver started at http://127.17.110.2:46323/ using document root <none> and password file <none>
I20260807 18:13:58.076920 18952 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:58.077088 18952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:58.077445 18952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:58.080801 18952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-1/data/instance:
uuid: "5682568e0da8465da4137f2cfed06898"
format_stamp: "Formatted at 2026-08-07 18:13:58 on dist-test-slave-39l8"
server_key: "7bb5894026ffff3361906eeb62c43590"
server_key_iv: "55009e28b0ecae6989c99bc2d16ec333"
server_key_version: "encryptionkey@0"
I20260807 18:13:58.081715 18952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance:
uuid: "5682568e0da8465da4137f2cfed06898"
format_stamp: "Formatted at 2026-08-07 18:13:58 on dist-test-slave-39l8"
server_key: "7bb5894026ffff3361906eeb62c43590"
server_key_iv: "55009e28b0ecae6989c99bc2d16ec333"
server_key_version: "encryptionkey@0"
I20260807 18:13:58.089069 18952 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260807 18:13:58.093741 18977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:58.095513 18952 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:13:58.095777 18952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-1/wal
uuid: "5682568e0da8465da4137f2cfed06898"
format_stamp: "Formatted at 2026-08-07 18:13:58 on dist-test-slave-39l8"
server_key: "7bb5894026ffff3361906eeb62c43590"
server_key_iv: "55009e28b0ecae6989c99bc2d16ec333"
server_key_version: "encryptionkey@0"
I20260807 18:13:58.095988 18952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:58.116724 18952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:58.121331 18952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:58.121596 18952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:58.122587 18952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:13:58.124089 18952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:13:58.124169 18952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:58.124259 18952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:13:58.124305 18952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:58.160230 18952 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.2:44995
I20260807 18:13:58.160337 19138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.2:44995 every 8 connection(s)
I20260807 18:13:58.161762 18952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
I20260807 18:13:58.167469 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 18952
I20260807 18:13:58.167699 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance
I20260807 18:13:58.168172 17848 external_mini_cluster.cc:1428] Setting key 519fa36a0cd5d5194bba44c148ee1fba
Aug 07 18:13:58 dist-test-slave-39l8 krb5kdc[18612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126438, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.110.3@KRBTEST.COM; defaulting to no policy
I20260807 18:13:58.193468 19143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:58.164580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:53147 (local address 127.17.110.62:34185)
0807 18:13:58.165101 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:58.165109 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:58.166195 (+ 1086us) server_negotiation.cc:407] Connection header received
0807 18:13:58.168655 (+ 2460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:58.168662 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:58.168773 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:58.168909 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:58.170669 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.171545 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.173555 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.174341 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.174492 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:58.184481 (+ 9989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:58.184493 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:58.184501 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:58.184578 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:58.189236 (+ 4658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:58.190298 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:58.190306 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:58.190310 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:58.190496 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:58.191103 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:58.191110 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:58.191114 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:58.191550 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:58.191755 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:58.192776 (+ 1021us) server_negotiation.cc:299] Negotiation successful
0807 18:13:58.193033 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":243,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.3@KRBTEST.COM" created.
I20260807 18:13:58.194836 19144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:58.165117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:34185 (local address 127.17.110.2:53147)
0807 18:13:58.165978 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:58.166042 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:58.168314 (+ 2272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:58.169199 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:58.169209 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:58.169818 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:58.170461 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:58.170481 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.171739 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.171743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:58.173360 (+ 1617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:58.173382 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.174753 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.174765 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:58.175155 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:58.176079 (+ 924us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:58.176129 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:58.184051 (+ 7922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:58.189491 (+ 5440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:58.189501 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:58.189519 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:58.190114 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:58.190670 (+ 556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:58.190673 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:58.190675 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:58.190953 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:58.191848 (+ 895us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:58.191856 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:58.192249 (+ 393us) client_negotiation.cc:769] Sending connection context
0807 18:13:58.192662 (+ 413us) client_negotiation.cc:240] Negotiation successful
0807 18:13:58.193284 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":361,"threads_started":1}
I20260807 18:13:58.196528 19139 heartbeater.cc:347] Connected to a master server at 127.17.110.62:34185
I20260807 18:13:58.196877 19139 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:58.197677 19139 heartbeater.cc:511] Master 127.17.110.62:34185 requested a full tablet report, sending...
I20260807 18:13:58.199427 18673 ts_manager.cc:194] Registered new tserver with Master: 5682568e0da8465da4137f2cfed06898 (127.17.110.2:44995)
I20260807 18:13:58.200497 18673 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.2@KRBTEST.COM'} at 127.17.110.2:53147
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
I20260807 18:13:58.223874 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.3:0
--local_ip_for_outbound_sockets=127.17.110.3
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:34185
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:42023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:13:58.605955 18945 heartbeater.cc:503] Master 127.17.110.62:34185 was elected leader, sending a full tablet report...
W20260807 18:13:58.609758 19148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:58.610143 19148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:58.610333 19148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:58.618492 19148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:58.619270 19148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:58.619549 19148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.3
I20260807 18:13:58.631387 19148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:42023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:34185
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.3
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:13:58.632843 19148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:58.634256 19148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:58.649165 19158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:58.649192 19163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:58.649392 19156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:58.650161 19148 server_base.cc:1034] running on GCE node
I20260807 18:13:58.651011 19148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:58.652158 19148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:58.653540 19148 hybrid_clock.cc:648] HybridClock initialized: now 1786126438653483 us; error 39 us; skew 500 ppm
I20260807 18:13:58.653957 19148 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:58 dist-test-slave-39l8 krb5kdc[18612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126438, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:58.657946 19148 init.cc:377] Logged in from keytab as kudu/127.17.110.3@KRBTEST.COM (short username kudu)
I20260807 18:13:58.660063 19148 webserver.cc:460] Webserver started at http://127.17.110.3:46391/ using document root <none> and password file <none>
I20260807 18:13:58.660928 19148 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:58.661293 19148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:58.662369 19148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:58.666206 19148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-2/data/instance:
uuid: "235818355a5a49499a00e0dd654746da"
format_stamp: "Formatted at 2026-08-07 18:13:58 on dist-test-slave-39l8"
server_key: "53a77e8f1a21f3c3f784ef7e8559f811"
server_key_iv: "d68b7644140df4067837b9d8e9a7c999"
server_key_version: "encryptionkey@0"
I20260807 18:13:58.667168 19148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance:
uuid: "235818355a5a49499a00e0dd654746da"
format_stamp: "Formatted at 2026-08-07 18:13:58 on dist-test-slave-39l8"
server_key: "53a77e8f1a21f3c3f784ef7e8559f811"
server_key_iv: "d68b7644140df4067837b9d8e9a7c999"
server_key_version: "encryptionkey@0"
I20260807 18:13:58.673377 19148 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 18:13:58.677119 19171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:58.678560 19148 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 18:13:58.678756 19148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-2/wal
uuid: "235818355a5a49499a00e0dd654746da"
format_stamp: "Formatted at 2026-08-07 18:13:58 on dist-test-slave-39l8"
server_key: "53a77e8f1a21f3c3f784ef7e8559f811"
server_key_iv: "d68b7644140df4067837b9d8e9a7c999"
server_key_version: "encryptionkey@0"
I20260807 18:13:58.678944 19148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:58.700780 19148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:58.705992 19148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:58.706288 19148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:58.707384 19148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:13:58.708823 19148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:13:58.708902 19148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:58.708992 19148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:13:58.709038 19148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:58.748668 19148 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.3:45017
I20260807 18:13:58.748736 19330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.3:45017 every 8 connection(s)
I20260807 18:13:58.750202 19148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
I20260807 18:13:58.757339 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 19148
I20260807 18:13:58.757584 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance
I20260807 18:13:58.758139 17848 external_mini_cluster.cc:1428] Setting key 798d54a5300bd9e9ddaec554af73d23b
Aug 07 18:13:58 dist-test-slave-39l8 krb5kdc[18612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126438, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:13:58.775688 19334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:58.753592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:56479 (local address 127.17.110.62:34185)
0807 18:13:58.754206 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:58.754212 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:58.755428 (+ 1216us) server_negotiation.cc:407] Connection header received
0807 18:13:58.757982 (+ 2554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:58.757987 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:58.758077 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:58.758184 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:58.760768 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.761997 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.763788 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.764215 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.764363 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:58.768323 (+ 3960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:58.768331 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:58.768338 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:58.768385 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:58.771507 (+ 3122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:58.772645 (+ 1138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:58.772652 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:58.772658 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:58.772824 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:58.773400 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:58.773406 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:58.773412 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:58.773742 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:58.773937 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:58.774814 (+ 877us) server_negotiation.cc:299] Negotiation successful
0807 18:13:58.775090 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":349,"threads_started":1}
I20260807 18:13:58.776821 19335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:58.754367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:34185 (local address 127.17.110.3:56479)
0807 18:13:58.755226 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:58.755267 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:58.757535 (+ 2268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:58.758611 (+ 1076us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:58.758635 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:58.759546 (+ 911us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:58.760539 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:58.760572 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.762182 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.762193 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:58.763611 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:58.763625 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.764482 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.764490 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:58.764744 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:58.765515 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:58.765546 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:58.768023 (+ 2477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:58.771681 (+ 3658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:58.771692 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:58.771712 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:58.772440 (+ 728us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:58.772972 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:58.772978 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:58.772983 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:58.773250 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:58.773902 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:58.773912 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:58.774341 (+ 429us) client_negotiation.cc:769] Sending connection context
0807 18:13:58.774671 (+ 330us) client_negotiation.cc:240] Negotiation successful
0807 18:13:58.775363 (+ 692us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":376,"threads_started":1}
I20260807 18:13:58.778373 19332 heartbeater.cc:347] Connected to a master server at 127.17.110.62:34185
I20260807 18:13:58.778766 19332 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:58.779575 19332 heartbeater.cc:511] Master 127.17.110.62:34185 requested a full tablet report, sending...
I20260807 18:13:58.781162 18673 ts_manager.cc:194] Registered new tserver with Master: 235818355a5a49499a00e0dd654746da (127.17.110.3:45017)
I20260807 18:13:58.782074 18673 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.3@KRBTEST.COM'} at 127.17.110.3:56479
I20260807 18:13:58.787590 17848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:13:58.810526 19334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:58.794887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48672 (local address 127.17.110.62:34185)
0807 18:13:58.795078 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:58.795084 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:58.795815 (+ 731us) server_negotiation.cc:407] Connection header received
0807 18:13:58.796037 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:58.796042 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:58.796157 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:58.796272 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:58.797994 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.799008 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.800710 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.801183 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.801366 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:58.804394 (+ 3028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:58.804399 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:58.804403 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:58.804445 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:58.807482 (+ 3037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:58.808217 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:58.808223 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:58.808226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:58.808370 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:58.808790 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:58.808798 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:58.808801 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:58.809225 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:58.809500 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:58.809859 (+ 359us) server_negotiation.cc:299] Negotiation successful
0807 18:13:58.810114 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:13:58.818231 18673 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48672:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:13:58.826977 18673 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:13:58.858796 19348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:58.842561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:38049 (local address 127.0.0.1:38272)
0807 18:13:58.848212 (+ 5651us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:58.848231 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:58.848373 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:58.849452 (+ 1079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:58.849460 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:58.849491 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:13:58.850020 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:58.850034 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.852152 (+ 2118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.852157 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:58.855122 (+ 2965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:58.855140 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.857496 (+ 2356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.857502 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:58.857809 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:58.857859 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 18:13:58.858015 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 18:13:58.858258 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5527,"mutex_wait_us":35,"thread_start_us":872,"threads_started":1}
I20260807 18:13:58.859133 19351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:58.843230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38272 (local address 127.17.110.1:38049)
0807 18:13:58.847779 (+ 4549us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:58.847789 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:58.848514 (+ 725us) server_negotiation.cc:407] Connection header received
0807 18:13:58.848627 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:58.848637 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:58.848948 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:58.849173 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:13:58.850250 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.851945 (+ 1695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.855335 (+ 3390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.856907 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.857051 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:58.857335 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:58.858068 (+ 733us) server_negotiation.cc:299] Negotiation successful
0807 18:13:58.858258 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4335,"thread_start_us":4826,"threads_started":1}
I20260807 18:13:58.861740 19353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:58.847351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:44995 (local address 127.0.0.1:34820)
0807 18:13:58.852853 (+ 5502us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:58.852874 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:58.853027 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:58.853897 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:58.853901 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:58.853926 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:13:58.854320 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:58.854329 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.856068 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.856076 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:58.858497 (+ 2421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:58.858510 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.860752 (+ 2242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.860758 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:58.860940 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:58.860973 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 18:13:58.861081 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 18:13:58.861255 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5348,"mutex_wait_us":1,"thread_start_us":5128,"threads_started":1}
I20260807 18:13:58.862322 19350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:58.842229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:45017 (local address 127.0.0.1:33084)
0807 18:13:58.843402 (+ 1173us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:58.843454 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:58.843630 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:58.852687 (+ 9057us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:58.852696 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:58.852726 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:13:58.853429 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:58.853450 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.855768 (+ 2318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.855774 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:58.858322 (+ 2548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:58.858339 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.861112 (+ 2773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.861119 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:58.861305 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:58.861346 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 18:13:58.861427 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 18:13:58.861603 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":912,"mutex_wait_us":93,"thread_start_us":5709,"threads_started":1}
I20260807 18:13:58.862556 19354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:58.847927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34820 (local address 127.17.110.2:44995)
0807 18:13:58.852233 (+ 4306us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:58.852245 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:58.853184 (+ 939us) server_negotiation.cc:407] Connection header received
0807 18:13:58.853294 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:58.853302 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:58.853558 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:58.853776 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:13:58.854547 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.855915 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.858681 (+ 2766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.860529 (+ 1848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.860691 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:58.860900 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:58.861992 (+ 1092us) server_negotiation.cc:299] Negotiation successful
0807 18:13:58.862256 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4142,"thread_start_us":4018,"threads_started":1}
I20260807 18:13:58.863409 19352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:58.843056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33084 (local address 127.17.110.3:45017)
0807 18:13:58.851847 (+ 8791us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:58.851858 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:58.851901 (+ 43us) server_negotiation.cc:407] Connection header received
0807 18:13:58.852038 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:58.852051 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:58.852407 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:58.853656 (+ 1249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:13:58.853995 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.855607 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.858552 (+ 2945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.860829 (+ 2277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.862599 (+ 1770us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:58.862921 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:58.863005 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 18:13:58.863135 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8581,"thread_start_us":8342,"threads_started":1}
I20260807 18:13:58.863631 18856 tablet_service.cc:1467] Processing CreateTablet for tablet d663e458568a4fc69ca14820d52f985f (DEFAULT_TABLE table=test-table [id=c08e2f3d7f194452a63047f731729033]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:13:58.865064 18856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d663e458568a4fc69ca14820d52f985f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:58.866174 19051 tablet_service.cc:1467] Processing CreateTablet for tablet d663e458568a4fc69ca14820d52f985f (DEFAULT_TABLE table=test-table [id=c08e2f3d7f194452a63047f731729033]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:13:58.868024 19051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d663e458568a4fc69ca14820d52f985f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:58.868103 19232 tablet_service.cc:1467] Processing CreateTablet for tablet d663e458568a4fc69ca14820d52f985f (DEFAULT_TABLE table=test-table [id=c08e2f3d7f194452a63047f731729033]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:13:58.869544 19232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d663e458568a4fc69ca14820d52f985f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:58.881330 19356 tablet_bootstrap.cc:492] T d663e458568a4fc69ca14820d52f985f P 5682568e0da8465da4137f2cfed06898: Bootstrap starting.
I20260807 18:13:58.884182 19357 tablet_bootstrap.cc:492] T d663e458568a4fc69ca14820d52f985f P c582a99a91b34101b4a741614ea2d531: Bootstrap starting.
I20260807 18:13:58.885778 19356 tablet_bootstrap.cc:654] T d663e458568a4fc69ca14820d52f985f P 5682568e0da8465da4137f2cfed06898: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:58.886106 19359 tablet_bootstrap.cc:492] T d663e458568a4fc69ca14820d52f985f P 235818355a5a49499a00e0dd654746da: Bootstrap starting.
I20260807 18:13:58.888027 19356 log.cc:824] T d663e458568a4fc69ca14820d52f985f P 5682568e0da8465da4137f2cfed06898: Log is configured to *not* fsync() on all Append() calls
I20260807 18:13:58.888464 19357 tablet_bootstrap.cc:654] T d663e458568a4fc69ca14820d52f985f P c582a99a91b34101b4a741614ea2d531: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:58.889478 19359 tablet_bootstrap.cc:654] T d663e458568a4fc69ca14820d52f985f P 235818355a5a49499a00e0dd654746da: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:58.890141 19357 log.cc:824] T d663e458568a4fc69ca14820d52f985f P c582a99a91b34101b4a741614ea2d531: Log is configured to *not* fsync() on all Append() calls
I20260807 18:13:58.890849 19359 log.cc:824] T d663e458568a4fc69ca14820d52f985f P 235818355a5a49499a00e0dd654746da: Log is configured to *not* fsync() on all Append() calls
I20260807 18:13:58.892452 19356 tablet_bootstrap.cc:492] T d663e458568a4fc69ca14820d52f985f P 5682568e0da8465da4137f2cfed06898: No bootstrap required, opened a new log
I20260807 18:13:58.892791 19356 ts_tablet_manager.cc:1397] T d663e458568a4fc69ca14820d52f985f P 5682568e0da8465da4137f2cfed06898: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260807 18:13:58.893267 19357 tablet_bootstrap.cc:492] T d663e458568a4fc69ca14820d52f985f P c582a99a91b34101b4a741614ea2d531: No bootstrap required, opened a new log
I20260807 18:13:58.893592 19357 ts_tablet_manager.cc:1397] T d663e458568a4fc69ca14820d52f985f P c582a99a91b34101b4a741614ea2d531: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260807 18:13:58.894107 19359 tablet_bootstrap.cc:492] T d663e458568a4fc69ca14820d52f985f P 235818355a5a49499a00e0dd654746da: No bootstrap required, opened a new log
I20260807 18:13:58.894407 19359 ts_tablet_manager.cc:1397] T d663e458568a4fc69ca14820d52f985f P 235818355a5a49499a00e0dd654746da: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 18:13:58.901921 19357 raft_consensus.cc:348] T d663e458568a4fc69ca14820d52f985f P c582a99a91b34101b4a741614ea2d531 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235818355a5a49499a00e0dd654746da" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 45017 } } peers { permanent_uuid: "c582a99a91b34101b4a741614ea2d531" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 38049 } } peers { permanent_uuid: "5682568e0da8465da4137f2cfed06898" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 44995 } }
I20260807 18:13:58.902355 19357 raft_consensus.cc:374] T d663e458568a4fc69ca14820d52f985f P c582a99a91b34101b4a741614ea2d531 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:58.902453 19357 raft_consensus.cc:729] T d663e458568a4fc69ca14820d52f985f P c582a99a91b34101b4a741614ea2d531 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c582a99a91b34101b4a741614ea2d531, State: Initialized, Role: FOLLOWER
I20260807 18:13:58.903141 19357 consensus_queue.cc:260] T d663e458568a4fc69ca14820d52f985f P c582a99a91b34101b4a741614ea2d531 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235818355a5a49499a00e0dd654746da" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 45017 } } peers { permanent_uuid: "c582a99a91b34101b4a741614ea2d531" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 38049 } } peers { permanent_uuid: "5682568e0da8465da4137f2cfed06898" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 44995 } }
I20260807 18:13:58.904878 19357 ts_tablet_manager.cc:1428] T d663e458568a4fc69ca14820d52f985f P c582a99a91b34101b4a741614ea2d531: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260807 18:13:58.904774 19356 raft_consensus.cc:348] T d663e458568a4fc69ca14820d52f985f P 5682568e0da8465da4137f2cfed06898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235818355a5a49499a00e0dd654746da" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 45017 } } peers { permanent_uuid: "c582a99a91b34101b4a741614ea2d531" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 38049 } } peers { permanent_uuid: "5682568e0da8465da4137f2cfed06898" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 44995 } }
I20260807 18:13:58.905263 19356 raft_consensus.cc:374] T d663e458568a4fc69ca14820d52f985f P 5682568e0da8465da4137f2cfed06898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:58.905388 19356 raft_consensus.cc:729] T d663e458568a4fc69ca14820d52f985f P 5682568e0da8465da4137f2cfed06898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5682568e0da8465da4137f2cfed06898, State: Initialized, Role: FOLLOWER
I20260807 18:13:58.906009 19356 consensus_queue.cc:260] T d663e458568a4fc69ca14820d52f985f P 5682568e0da8465da4137f2cfed06898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235818355a5a49499a00e0dd654746da" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 45017 } } peers { permanent_uuid: "c582a99a91b34101b4a741614ea2d531" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 38049 } } peers { permanent_uuid: "5682568e0da8465da4137f2cfed06898" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 44995 } }
I20260807 18:13:58.906427 19359 raft_consensus.cc:348] T d663e458568a4fc69ca14820d52f985f P 235818355a5a49499a00e0dd654746da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235818355a5a49499a00e0dd654746da" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 45017 } } peers { permanent_uuid: "c582a99a91b34101b4a741614ea2d531" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 38049 } } peers { permanent_uuid: "5682568e0da8465da4137f2cfed06898" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 44995 } }
I20260807 18:13:58.906844 19359 raft_consensus.cc:374] T d663e458568a4fc69ca14820d52f985f P 235818355a5a49499a00e0dd654746da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:58.906963 19359 raft_consensus.cc:729] T d663e458568a4fc69ca14820d52f985f P 235818355a5a49499a00e0dd654746da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235818355a5a49499a00e0dd654746da, State: Initialized, Role: FOLLOWER
I20260807 18:13:58.907565 19359 consensus_queue.cc:260] T d663e458568a4fc69ca14820d52f985f P 235818355a5a49499a00e0dd654746da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235818355a5a49499a00e0dd654746da" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 45017 } } peers { permanent_uuid: "c582a99a91b34101b4a741614ea2d531" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 38049 } } peers { permanent_uuid: "5682568e0da8465da4137f2cfed06898" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 44995 } }
I20260807 18:13:58.908756 19332 heartbeater.cc:503] Master 127.17.110.62:34185 was elected leader, sending a full tablet report...
I20260807 18:13:58.909449 19359 ts_tablet_manager.cc:1428] T d663e458568a4fc69ca14820d52f985f P 235818355a5a49499a00e0dd654746da: Time spent starting tablet: real 0.015s user 0.011s sys 0.003s
I20260807 18:13:58.909752 19139 heartbeater.cc:503] Master 127.17.110.62:34185 was elected leader, sending a full tablet report...
I20260807 18:13:58.911267 19356 ts_tablet_manager.cc:1428] T d663e458568a4fc69ca14820d52f985f P 5682568e0da8465da4137f2cfed06898: Time spent starting tablet: real 0.018s user 0.016s sys 0.001s
W20260807 18:13:58.914017 19140 tablet.cc:2369] T d663e458568a4fc69ca14820d52f985f P 5682568e0da8465da4137f2cfed06898: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:13:58.958141 19370 raft_consensus.cc:482] T d663e458568a4fc69ca14820d52f985f P 235818355a5a49499a00e0dd654746da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:13:58.958528 19370 raft_consensus.cc:504] T d663e458568a4fc69ca14820d52f985f P 235818355a5a49499a00e0dd654746da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235818355a5a49499a00e0dd654746da" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 45017 } } peers { permanent_uuid: "c582a99a91b34101b4a741614ea2d531" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 38049 } } peers { permanent_uuid: "5682568e0da8465da4137f2cfed06898" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 44995 } }
I20260807 18:13:58.960393 19370 leader_election.cc:288] T d663e458568a4fc69ca14820d52f985f P 235818355a5a49499a00e0dd654746da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c582a99a91b34101b4a741614ea2d531 (127.17.110.1:38049), 5682568e0da8465da4137f2cfed06898 (127.17.110.2:44995)
I20260807 18:13:58.966746 19335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:58.960921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:38049 (local address 127.17.110.3:59731)
0807 18:13:58.961175 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:58.961202 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:58.961327 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:58.961765 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:58.961770 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:58.961800 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:13:58.962037 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:58.962048 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.963307 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.963311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:58.964643 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:58.964659 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.965900 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.965906 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:58.966117 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:58.966144 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 18:13:58.966261 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 18:13:58.966415 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260807 18:13:58.966748 19351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:58.961067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:59731 (local address 127.17.110.1:38049)
0807 18:13:58.961338 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:58.961345 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:58.961369 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:13:58.961485 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:58.961492 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:58.961599 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:58.961720 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:13:58.962212 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.963124 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.964823 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.965715 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.965902 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:58.965980 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:58.966299 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 18:13:58.966437 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260807 18:13:58.967938 19354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:58.961803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:57567 (local address 127.17.110.2:44995)
0807 18:13:58.962069 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:58.962078 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:58.962531 (+ 453us) server_negotiation.cc:407] Connection header received
0807 18:13:58.962787 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:58.962791 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:58.962853 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:58.962962 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:13:58.963595 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.964307 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.966044 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.967068 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.967268 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:58.967352 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:58.967641 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 18:13:58.967751 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:13:58.968122 19372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:58.961437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:44995 (local address 127.17.110.3:57567)
0807 18:13:58.962420 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:58.962436 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:58.962601 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:58.962985 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:58.962989 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:58.963016 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:13:58.963422 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:58.963434 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.964482 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.964486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:58.965855 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:58.965867 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.967307 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.967312 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:58.967434 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:58.967459 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 18:13:58.967601 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 18:13:58.967753 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":883,"thread_start_us":648,"threads_started":1}
I20260807 18:13:58.968250 18886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d663e458568a4fc69ca14820d52f985f" candidate_uuid: "235818355a5a49499a00e0dd654746da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c582a99a91b34101b4a741614ea2d531" is_pre_election: true
I20260807 18:13:58.969055 18886 raft_consensus.cc:2454] T d663e458568a4fc69ca14820d52f985f P c582a99a91b34101b4a741614ea2d531 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 235818355a5a49499a00e0dd654746da in term 0.
I20260807 18:13:58.969492 19076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d663e458568a4fc69ca14820d52f985f" candidate_uuid: "235818355a5a49499a00e0dd654746da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5682568e0da8465da4137f2cfed06898" is_pre_election: true
I20260807 18:13:58.969920 19172 leader_election.cc:302] T d663e458568a4fc69ca14820d52f985f P 235818355a5a49499a00e0dd654746da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235818355a5a49499a00e0dd654746da, c582a99a91b34101b4a741614ea2d531; no voters:
I20260807 18:13:58.970096 19076 raft_consensus.cc:2454] T d663e458568a4fc69ca14820d52f985f P 5682568e0da8465da4137f2cfed06898 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 235818355a5a49499a00e0dd654746da in term 0.
I20260807 18:13:58.970444 19370 raft_consensus.cc:2781] T d663e458568a4fc69ca14820d52f985f P 235818355a5a49499a00e0dd654746da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:13:58.970649 19370 raft_consensus.cc:482] T d663e458568a4fc69ca14820d52f985f P 235818355a5a49499a00e0dd654746da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:13:58.970813 19370 raft_consensus.cc:3037] T d663e458568a4fc69ca14820d52f985f P 235818355a5a49499a00e0dd654746da [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:58.973286 19370 raft_consensus.cc:504] T d663e458568a4fc69ca14820d52f985f P 235818355a5a49499a00e0dd654746da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235818355a5a49499a00e0dd654746da" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 45017 } } peers { permanent_uuid: "c582a99a91b34101b4a741614ea2d531" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 38049 } } peers { permanent_uuid: "5682568e0da8465da4137f2cfed06898" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 44995 } }
I20260807 18:13:58.973959 19370 leader_election.cc:288] T d663e458568a4fc69ca14820d52f985f P 235818355a5a49499a00e0dd654746da [CANDIDATE]: Term 1 election: Requested vote from peers c582a99a91b34101b4a741614ea2d531 (127.17.110.1:38049), 5682568e0da8465da4137f2cfed06898 (127.17.110.2:44995)
I20260807 18:13:58.974618 18886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d663e458568a4fc69ca14820d52f985f" candidate_uuid: "235818355a5a49499a00e0dd654746da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c582a99a91b34101b4a741614ea2d531"
I20260807 18:13:58.974623 19076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d663e458568a4fc69ca14820d52f985f" candidate_uuid: "235818355a5a49499a00e0dd654746da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5682568e0da8465da4137f2cfed06898"
I20260807 18:13:58.974889 19076 raft_consensus.cc:3037] T d663e458568a4fc69ca14820d52f985f P 5682568e0da8465da4137f2cfed06898 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:58.974892 18886 raft_consensus.cc:3037] T d663e458568a4fc69ca14820d52f985f P c582a99a91b34101b4a741614ea2d531 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:58.977450 18886 raft_consensus.cc:2454] T d663e458568a4fc69ca14820d52f985f P c582a99a91b34101b4a741614ea2d531 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235818355a5a49499a00e0dd654746da in term 1.
I20260807 18:13:58.977660 19076 raft_consensus.cc:2454] T d663e458568a4fc69ca14820d52f985f P 5682568e0da8465da4137f2cfed06898 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235818355a5a49499a00e0dd654746da in term 1.
I20260807 18:13:58.978044 19172 leader_election.cc:302] T d663e458568a4fc69ca14820d52f985f P 235818355a5a49499a00e0dd654746da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235818355a5a49499a00e0dd654746da, c582a99a91b34101b4a741614ea2d531; no voters:
I20260807 18:13:58.978427 19370 raft_consensus.cc:2781] T d663e458568a4fc69ca14820d52f985f P 235818355a5a49499a00e0dd654746da [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:13:58.979012 19370 raft_consensus.cc:686] T d663e458568a4fc69ca14820d52f985f P 235818355a5a49499a00e0dd654746da [term 1 LEADER]: Becoming Leader. State: Replica: 235818355a5a49499a00e0dd654746da, State: Running, Role: LEADER
I20260807 18:13:58.979547 19370 consensus_queue.cc:237] T d663e458568a4fc69ca14820d52f985f P 235818355a5a49499a00e0dd654746da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235818355a5a49499a00e0dd654746da" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 45017 } } peers { permanent_uuid: "c582a99a91b34101b4a741614ea2d531" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 38049 } } peers { permanent_uuid: "5682568e0da8465da4137f2cfed06898" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 44995 } }
I20260807 18:13:58.985024 18672 catalog_manager.cc:5495] T d663e458568a4fc69ca14820d52f985f P 235818355a5a49499a00e0dd654746da reported cstate change: term changed from 0 to 1, leader changed from <none> to 235818355a5a49499a00e0dd654746da (127.17.110.3). New cstate: current_term: 1 leader_uuid: "235818355a5a49499a00e0dd654746da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235818355a5a49499a00e0dd654746da" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 45017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c582a99a91b34101b4a741614ea2d531" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5682568e0da8465da4137f2cfed06898" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 44995 } health_report { overall_health: UNKNOWN } } }
W20260807 18:13:59.002880 19333 tablet.cc:2369] T d663e458568a4fc69ca14820d52f985f P 235818355a5a49499a00e0dd654746da: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 18:13:59 dist-test-slave-39l8 krb5kdc[18612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.1@KRBTEST.COM
I20260807 18:13:59.026479 19351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:59.008578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38284 (local address 127.17.110.1:38049)
0807 18:13:59.008897 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:59.008904 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:59.009220 (+ 316us) server_negotiation.cc:407] Connection header received
0807 18:13:59.009390 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:59.009395 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:59.009485 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:59.009584 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:59.011422 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:59.012292 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:59.013634 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:59.014130 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:59.014244 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:59.018835 (+ 4591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:59.018855 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:59.018874 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:59.018924 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:59.023514 (+ 4590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:59.024345 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:59.024358 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:59.024361 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:59.024499 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:59.024906 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:59.024909 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:59.024911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:59.025293 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:59.025565 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:59.025875 (+ 310us) server_negotiation.cc:299] Negotiation successful
0807 18:13:59.026052 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
Aug 07 18:13:59 dist-test-slave-39l8 krb5kdc[18612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126439, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260807 18:13:59.067939 18946 tablet.cc:2369] T d663e458568a4fc69ca14820d52f985f P c582a99a91b34101b4a741614ea2d531: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 18:13:59 dist-test-slave-39l8 krb5kdc[18612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126439, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.110.1@KRBTEST.COM
I20260807 18:13:59.082988 19351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:59.066845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38298 (local address 127.17.110.1:38049)
0807 18:13:59.067069 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:59.067075 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:59.067299 (+ 224us) server_negotiation.cc:407] Connection header received
0807 18:13:59.067464 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:59.067469 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:59.067552 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:59.067682 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:59.069737 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:59.070510 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:59.071842 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:59.072279 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:59.072386 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:59.076636 (+ 4250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:59.076642 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:59.076645 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:59.076689 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:59.079535 (+ 2846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:59.080213 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:59.080226 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:59.080229 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:59.080385 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:59.081051 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:59.081057 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:59.081061 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:59.081421 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:59.081846 (+ 425us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:59.082210 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 18:13:59.082483 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Aug 07 18:13:59 dist-test-slave-39l8 krb5kdc[18612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 18:13:59 dist-test-slave-39l8 krb5kdc[18612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.1@KRBTEST.COM
I20260807 18:13:59.138569 19351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:59.123248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38312 (local address 127.17.110.1:38049)
0807 18:13:59.123529 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:59.123537 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:59.123862 (+ 325us) server_negotiation.cc:407] Connection header received
0807 18:13:59.124093 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:59.124098 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:59.124164 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:59.124280 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:59.125722 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:59.126416 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:59.127840 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:59.128287 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:59.128424 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:59.132977 (+ 4553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:59.132983 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:59.132986 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:59.133024 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:59.135562 (+ 2538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:59.136260 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:59.136266 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:59.136270 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:59.136398 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:59.136955 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:59.136961 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:59.136965 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:59.137315 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:59.137647 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:59.137930 (+ 283us) server_negotiation.cc:299] Negotiation successful
0807 18:13:59.138200 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260807 18:13:59.140518 18825 tablet_service.cc:2908] Rejecting scan request for tablet d663e458568a4fc69ca14820d52f985f: Uninitialized: clean time has not yet been initialized
I20260807 18:13:59.144150 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 18763
I20260807 18:13:59.158288 19351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:59.145150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38328 (local address 127.17.110.1:38049)
0807 18:13:59.145314 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:59.145319 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:59.145741 (+ 422us) server_negotiation.cc:407] Connection header received
0807 18:13:59.145920 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:59.145924 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:59.145999 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:59.146082 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:59.147329 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:59.148069 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:59.149552 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:59.149974 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:59.150078 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:59.152553 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:59.152558 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:59.152561 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:59.152592 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:59.155622 (+ 3030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:59.156338 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:59.156348 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:59.156351 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:59.156474 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:59.156865 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:59.156870 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:59.156873 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:59.157251 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:59.157586 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:59.157857 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 18:13:59.158024 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 18:13:59.159129 18939 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:59.256644 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 18763
I20260807 18:13:59.283567 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 18952
Aug 07 18:13:59 dist-test-slave-39l8 krb5kdc[18612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.2@KRBTEST.COM
I20260807 18:13:59.299495 19354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:59.284469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34826 (local address 127.17.110.2:44995)
0807 18:13:59.284694 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:59.284703 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:59.284726 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:13:59.284797 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:59.284801 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:59.284864 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:59.284994 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:59.286285 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:59.287204 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:59.288556 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:59.288977 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:59.289130 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:59.293461 (+ 4331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:59.293484 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:59.293506 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:59.293552 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:59.296489 (+ 2937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:59.297209 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:59.297226 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:59.297229 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:59.297363 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:59.297789 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:59.297794 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:59.297797 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:59.298220 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:59.298584 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:59.298886 (+ 302us) server_negotiation.cc:299] Negotiation successful
0807 18:13:59.299157 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:13:59.300388 19133 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:59.393471 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 18952
W20260807 18:13:59.418349 19172 connection.cc:535] client connection to 127.17.110.2:44995 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 18:13:59.418694 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 19148
W20260807 18:13:59.418870 19172 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 18:13:59.420737 19372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:59.420237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:44995 (local address 127.17.110.3:59169)
0807 18:13:59.420415 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:59.420548 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.2:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260807 18:13:59.421084 19172 consensus_peers.cc:487] T d663e458568a4fc69ca14820d52f985f P 235818355a5a49499a00e0dd654746da -> Peer 5682568e0da8465da4137f2cfed06898 (127.17.110.2:44995): Couldn't send request to peer 5682568e0da8465da4137f2cfed06898. Status: Network error: Client connection negotiation failed: client connection to 127.17.110.2:44995: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
Aug 07 18:13:59 dist-test-slave-39l8 krb5kdc[18612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.3@KRBTEST.COM
I20260807 18:13:59.436522 19395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:59.419702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33086 (local address 127.17.110.3:45017)
0807 18:13:59.420234 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:59.420238 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:59.420296 (+ 58us) server_negotiation.cc:407] Connection header received
0807 18:13:59.420593 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:59.420600 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:59.420681 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:59.420832 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:59.422231 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:59.423094 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:59.424414 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:59.424842 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:59.424947 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:59.429714 (+ 4767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:59.429732 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:59.429750 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:59.429808 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:59.433345 (+ 3537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:59.434113 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:59.434122 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:59.434125 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:59.434296 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:59.434654 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:59.434658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:59.434661 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:59.435273 (+ 612us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:59.435535 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:59.435829 (+ 294us) server_negotiation.cc:299] Negotiation successful
0807 18:13:59.436066 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":266,"threads_started":1}
I20260807 18:13:59.437325 19323 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:59.530864 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 19148
I20260807 18:13:59.557623 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 18632
I20260807 18:13:59.558992 18718 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:59.685199 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 18632
2026-08-07T18:13:59Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4957 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:13:59 dist-test-slave-39l8 krb5kdc[19400](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:13:59 dist-test-slave-39l8 krb5kdc[19400](info): set up 2 sockets
Aug 07 18:13:59 dist-test-slave-39l8 krb5kdc[19400](info): commencing operation
krb5kdc: starting...
W20260807 18:14:01.044448 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.305s user 0.002s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:14:01 dist-test-slave-39l8 krb5kdc[19400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:14:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:14:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.110.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
I20260807 18:14:01.279624 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:34857
--webserver_interface=127.17.110.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:39065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:34857
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:14:01.642010 19418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:01.642334 19418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:01.642509 19418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:01.649955 19418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:14:01.650104 19418 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 18:14:01.650149 19418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:01.650274 19418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:01.650352 19418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:14:01.650446 19418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:14:01.660905 19418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:39065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:34857
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:34857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:01.662343 19418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:01.663832 19418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:01.676424 19426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:01.676405 19430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:01.676623 19427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:01.677320 19418 server_base.cc:1034] running on GCE node
I20260807 18:14:01.678252 19418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:01.679929 19418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:01.681218 19418 hybrid_clock.cc:648] HybridClock initialized: now 1786126441681144 us; error 49 us; skew 500 ppm
I20260807 18:14:01.681612 19418 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:01 dist-test-slave-39l8 krb5kdc[19400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126441, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:01.685165 19418 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:14:01.686941 19418 webserver.cc:460] Webserver started at http://127.17.110.62:36839/ using document root <none> and password file <none>
I20260807 18:14:01.687714 19418 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:01.687868 19418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:01.688221 19418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:01.691385 19418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "ca311ba787804c1b9e854c216d9e7289"
format_stamp: "Formatted at 2026-08-07 18:14:01 on dist-test-slave-39l8"
server_key: "898dde30614926f46455eb85358f460e"
server_key_iv: "d6155c82c935f1fab9684c446e28928a"
server_key_version: "encryptionkey@0"
I20260807 18:14:01.692292 19418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "ca311ba787804c1b9e854c216d9e7289"
format_stamp: "Formatted at 2026-08-07 18:14:01 on dist-test-slave-39l8"
server_key: "898dde30614926f46455eb85358f460e"
server_key_iv: "d6155c82c935f1fab9684c446e28928a"
server_key_version: "encryptionkey@0"
I20260807 18:14:01.698036 19418 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 18:14:01.702015 19437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:01.703569 19418 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 18:14:01.703778 19418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "ca311ba787804c1b9e854c216d9e7289"
format_stamp: "Formatted at 2026-08-07 18:14:01 on dist-test-slave-39l8"
server_key: "898dde30614926f46455eb85358f460e"
server_key_iv: "d6155c82c935f1fab9684c446e28928a"
server_key_version: "encryptionkey@0"
I20260807 18:14:01.704035 19418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:01.745674 19418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:01.750288 19418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:01.750604 19418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:01.775291 19418 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:34857
I20260807 18:14:01.775285 19524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:34857 every 8 connection(s)
I20260807 18:14:01.776867 19418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:14:01.781080 19525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:01.783973 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 19418
I20260807 18:14:01.784258 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:14:01.785174 17848 external_mini_cluster.cc:1428] Setting key a3a7f41a4b630cde4e7fc1af1fa56c24
I20260807 18:14:01.796479 19525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca311ba787804c1b9e854c216d9e7289: Bootstrap starting.
I20260807 18:14:01.801282 19525 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca311ba787804c1b9e854c216d9e7289: Neither blocks nor log segments found. Creating new log.
Aug 07 18:14:01 dist-test-slave-39l8 krb5kdc[19400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:01.802768 19525 log.cc:824] T 00000000000000000000000000000000 P ca311ba787804c1b9e854c216d9e7289: Log is configured to *not* fsync() on all Append() calls
I20260807 18:14:01.806228 19525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca311ba787804c1b9e854c216d9e7289: No bootstrap required, opened a new log
I20260807 18:14:01.813202 19529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:01.787738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45518 (local address 127.17.110.62:34857)
0807 18:14:01.788691 (+ 953us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:01.788704 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:01.788752 (+ 48us) server_negotiation.cc:407] Connection header received
0807 18:14:01.790553 (+ 1801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:01.790588 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:01.791108 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:01.791456 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:01.793611 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:01.795105 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:01.797125 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:01.797872 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:01.798028 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:01.803961 (+ 5933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:01.803972 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:01.803993 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:01.804048 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:01.808057 (+ 4009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:01.808985 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:01.808996 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:01.809006 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:01.809302 (+ 296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:01.809764 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:01.809770 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:01.809774 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:01.810490 (+ 716us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:01.810878 (+ 388us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:01.811424 (+ 546us) server_negotiation.cc:299] Negotiation successful
0807 18:14:01.811983 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":460,"threads_started":1}
I20260807 18:14:01.819454 19525 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca311ba787804c1b9e854c216d9e7289 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca311ba787804c1b9e854c216d9e7289" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 34857 } }
I20260807 18:14:01.819975 19525 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca311ba787804c1b9e854c216d9e7289 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:01.820164 19525 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca311ba787804c1b9e854c216d9e7289 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca311ba787804c1b9e854c216d9e7289, State: Initialized, Role: FOLLOWER
I20260807 18:14:01.821049 19525 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca311ba787804c1b9e854c216d9e7289 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca311ba787804c1b9e854c216d9e7289" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 34857 } }
I20260807 18:14:01.821357 19525 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca311ba787804c1b9e854c216d9e7289 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:14:01.821491 19525 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca311ba787804c1b9e854c216d9e7289 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:14:01.821641 19525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca311ba787804c1b9e854c216d9e7289 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:01.853291 19525 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca311ba787804c1b9e854c216d9e7289 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca311ba787804c1b9e854c216d9e7289" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 34857 } }
I20260807 18:14:01.854310 19525 leader_election.cc:302] T 00000000000000000000000000000000 P ca311ba787804c1b9e854c216d9e7289 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca311ba787804c1b9e854c216d9e7289; no voters:
I20260807 18:14:01.855173 19525 leader_election.cc:288] T 00000000000000000000000000000000 P ca311ba787804c1b9e854c216d9e7289 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:14:01.855345 19533 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca311ba787804c1b9e854c216d9e7289 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:14:01.857133 19533 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca311ba787804c1b9e854c216d9e7289 [term 1 LEADER]: Becoming Leader. State: Replica: ca311ba787804c1b9e854c216d9e7289, State: Running, Role: LEADER
I20260807 18:14:01.857929 19525 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca311ba787804c1b9e854c216d9e7289 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:14:01.857918 19533 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca311ba787804c1b9e854c216d9e7289 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca311ba787804c1b9e854c216d9e7289" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 34857 } }
I20260807 18:14:01.861614 19535 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca311ba787804c1b9e854c216d9e7289 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca311ba787804c1b9e854c216d9e7289. Latest consensus state: current_term: 1 leader_uuid: "ca311ba787804c1b9e854c216d9e7289" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca311ba787804c1b9e854c216d9e7289" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 34857 } } }
I20260807 18:14:01.862838 19535 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca311ba787804c1b9e854c216d9e7289 [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:01.861747 19534 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca311ba787804c1b9e854c216d9e7289 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca311ba787804c1b9e854c216d9e7289" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca311ba787804c1b9e854c216d9e7289" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 34857 } } }
I20260807 18:14:01.863426 19534 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca311ba787804c1b9e854c216d9e7289 [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:01.866040 19543 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:14:01.874235 19543 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:14:01.887002 19543 catalog_manager.cc:1347] Generated new cluster ID: 7b412fa4eb554acc8971236f83a4e3fc
I20260807 18:14:01.887177 19543 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:14:01.906338 19543 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:14:01.907923 19543 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:14:01.919921 19543 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca311ba787804c1b9e854c216d9e7289: Generated new TSK 0
I20260807 18:14:01.920960 19543 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.110.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
W20260807 18:14:02.745807 17848 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.17.110.1: real 0.817s user 0.000s sys 0.013s
I20260807 18:14:02.747099 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.1:0
--local_ip_for_outbound_sockets=127.17.110.1
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:34857
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:39065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:14:03.148823 19561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:03.149251 19561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:03.149417 19561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:03.157908 19561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:03.158057 19561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:03.158201 19561 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:14:03.158339 19561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.1
I20260807 18:14:03.170648 19561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:39065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.110.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.110.62:34857
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.1
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:03.172207 19561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:03.173748 19561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:03.189050 19568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:03.189077 19569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:03.189296 19571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:03.189561 19561 server_base.cc:1034] running on GCE node
I20260807 18:14:03.190651 19561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:03.191774 19561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:03.193109 19561 hybrid_clock.cc:648] HybridClock initialized: now 1786126443192971 us; error 82 us; skew 500 ppm
I20260807 18:14:03.193480 19561 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:03 dist-test-slave-39l8 krb5kdc[19400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126443, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:03.197600 19561 init.cc:377] Logged in from keytab as kudu/127.17.110.1@KRBTEST.COM (short username kudu)
I20260807 18:14:03.199529 19561 webserver.cc:460] Webserver started at http://127.17.110.1:39605/ using document root <none> and password file <none>
I20260807 18:14:03.200385 19561 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:03.200549 19561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:03.200910 19561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:03.204304 19561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-0/data/instance:
uuid: "1131279766ec49aba2aae37aa315c5a7"
format_stamp: "Formatted at 2026-08-07 18:14:03 on dist-test-slave-39l8"
server_key: "4dba08c507a1580e0527184d36e47fe9"
server_key_iv: "5ccd3cc01add86048d1469a2386c7258"
server_key_version: "encryptionkey@0"
I20260807 18:14:03.205302 19561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance:
uuid: "1131279766ec49aba2aae37aa315c5a7"
format_stamp: "Formatted at 2026-08-07 18:14:03 on dist-test-slave-39l8"
server_key: "4dba08c507a1580e0527184d36e47fe9"
server_key_iv: "5ccd3cc01add86048d1469a2386c7258"
server_key_version: "encryptionkey@0"
I20260807 18:14:03.212172 19561 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 18:14:03.216871 19580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:03.218999 19561 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.004s
I20260807 18:14:03.219267 19561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-0/wal
uuid: "1131279766ec49aba2aae37aa315c5a7"
format_stamp: "Formatted at 2026-08-07 18:14:03 on dist-test-slave-39l8"
server_key: "4dba08c507a1580e0527184d36e47fe9"
server_key_iv: "5ccd3cc01add86048d1469a2386c7258"
server_key_version: "encryptionkey@0"
I20260807 18:14:03.219581 19561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:03.244555 19561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:03.250063 19561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:03.250429 19561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:03.253698 19561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:14:03.253847 19561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:03.254022 19561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:14:03.254120 19561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:14:03 dist-test-slave-39l8 krb5kdc[19400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126443, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:03.282232 19591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:03.254880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:44107 (local address 127.17.110.62:34857)
0807 18:14:03.255421 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:03.255428 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:03.256977 (+ 1549us) server_negotiation.cc:407] Connection header received
0807 18:14:03.259692 (+ 2715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:03.259703 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:03.259839 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:03.259994 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:03.263033 (+ 3039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:03.263850 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:03.265854 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:03.266398 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:03.266562 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:03.272834 (+ 6272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:03.272844 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:03.272849 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:03.272922 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:03.277050 (+ 4128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:03.278412 (+ 1362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:03.278421 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:03.278423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:03.278611 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:03.279405 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:03.279413 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:03.279416 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:03.279876 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:03.280066 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:03.281222 (+ 1156us) server_negotiation.cc:299] Negotiation successful
0807 18:14:03.281583 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":257,"threads_started":1}
I20260807 18:14:03.283490 19592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:03.255694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:34857 (local address 127.17.110.1:44107)
0807 18:14:03.256724 (+ 1030us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:03.256797 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:03.259206 (+ 2409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:03.260376 (+ 1170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:03.260397 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:03.261423 (+ 1026us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:03.262674 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:03.262753 (+ 79us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:03.264081 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:03.264087 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:03.265649 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:03.265667 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:03.266787 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:03.266801 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:03.267290 (+ 489us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:03.268424 (+ 1134us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:03.268464 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:03.272357 (+ 3893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:03.277274 (+ 4917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:03.277295 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:03.277340 (+ 45us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:03.278159 (+ 819us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:03.278813 (+ 654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:03.278822 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:03.278827 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:03.279223 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:03.280045 (+ 822us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:03.280055 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:03.280654 (+ 599us) client_negotiation.cc:769] Sending connection context
0807 18:14:03.281001 (+ 347us) client_negotiation.cc:240] Negotiation successful
0807 18:14:03.281735 (+ 734us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"thread_start_us":419,"threads_started":1}
I20260807 18:14:03.311316 19591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:03.294448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:42237 (local address 127.17.110.62:34857)
0807 18:14:03.294677 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:03.294686 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:03.295593 (+ 907us) server_negotiation.cc:407] Connection header received
0807 18:14:03.295744 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:03.295753 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:03.295875 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:03.296021 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:03.297081 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:03.298653 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:03.301026 (+ 2373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:03.301510 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:03.301686 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:03.304457 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:03.304463 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:03.304466 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:03.304503 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:03.308132 (+ 3629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:03.309103 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:03.309109 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:03.309113 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:03.309273 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:03.309681 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:03.309686 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:03.309689 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:03.310057 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:03.310226 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:03.310675 (+ 449us) server_negotiation.cc:299] Negotiation successful
0807 18:14:03.310956 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 18:14:03.311573 19710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:03.294552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:34857 (local address 127.17.110.1:42237)
0807 18:14:03.295452 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:03.295470 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:03.295614 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:03.296071 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:03.296074 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:03.296577 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:03.296901 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:03.296912 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:03.298903 (+ 1991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:03.298912 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:03.300807 (+ 1895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:03.300826 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:03.301734 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:03.301740 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:03.301919 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:03.302639 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:03.302660 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:03.304127 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:03.308311 (+ 4184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:03.308318 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:03.308322 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:03.308939 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:03.309400 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:03.309406 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:03.309410 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:03.309561 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:03.310186 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:03.310192 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:03.310399 (+ 207us) client_negotiation.cc:769] Sending connection context
0807 18:14:03.310555 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 18:14:03.311024 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"thread_start_us":523,"threads_started":1}
I20260807 18:14:03.318944 19561 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.1:35811
I20260807 18:14:03.318974 19764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.1:35811 every 8 connection(s)
I20260807 18:14:03.320356 19561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
I20260807 18:14:03.323107 19765 heartbeater.cc:347] Connected to a master server at 127.17.110.62:34857
I20260807 18:14:03.323429 19765 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:03.323845 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 19561
I20260807 18:14:03.324146 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance
I20260807 18:14:03.324353 19765 heartbeater.cc:511] Master 127.17.110.62:34857 requested a full tablet report, sending...
I20260807 18:14:03.324949 17848 external_mini_cluster.cc:1428] Setting key 679022ef2d8b72242f0d32671cce55c3
I20260807 18:14:03.327657 19459 ts_manager.cc:194] Registered new tserver with Master: 1131279766ec49aba2aae37aa315c5a7 (127.17.110.1:35811)
I20260807 18:14:03.330066 19459 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.1@KRBTEST.COM'} at 127.17.110.1:44107
WARNING: no policy specified for kudu/127.17.110.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
W20260807 18:14:03.705873 17848 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.17.110.2: real 0.377s user 0.000s sys 0.013s
I20260807 18:14:03.706760 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.2:0
--local_ip_for_outbound_sockets=127.17.110.2
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:34857
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:39065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:14:04.068174 19775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:04.068532 19775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:04.068730 19775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:04.076735 19775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:04.076867 19775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:04.077060 19775 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:14:04.077209 19775 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.2
I20260807 18:14:04.088075 19775 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:39065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.110.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.110.62:34857
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.2
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:04.089534 19775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:04.091156 19775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:04.106848 19783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:04.107129 19784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:04.106856 19787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:04.107647 19775 server_base.cc:1034] running on GCE node
I20260807 18:14:04.108417 19775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:04.109580 19775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:04.110864 19775 hybrid_clock.cc:648] HybridClock initialized: now 1786126444110740 us; error 78 us; skew 500 ppm
I20260807 18:14:04.111248 19775 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:04 dist-test-slave-39l8 krb5kdc[19400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126444, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:04.119522 19775 init.cc:377] Logged in from keytab as kudu/127.17.110.2@KRBTEST.COM (short username kudu)
I20260807 18:14:04.121296 19775 webserver.cc:460] Webserver started at http://127.17.110.2:37831/ using document root <none> and password file <none>
I20260807 18:14:04.122071 19775 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:04.122174 19775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:04.122462 19775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:04.125921 19775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-1/data/instance:
uuid: "aafbbd4e646344f5a4e257344c5f9ddd"
format_stamp: "Formatted at 2026-08-07 18:14:04 on dist-test-slave-39l8"
server_key: "462389bf7fd8ecb5e8a45586ca5f3b68"
server_key_iv: "a221c6cee483b21bf918c92b4e14c67e"
server_key_version: "encryptionkey@0"
I20260807 18:14:04.126966 19775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance:
uuid: "aafbbd4e646344f5a4e257344c5f9ddd"
format_stamp: "Formatted at 2026-08-07 18:14:04 on dist-test-slave-39l8"
server_key: "462389bf7fd8ecb5e8a45586ca5f3b68"
server_key_iv: "a221c6cee483b21bf918c92b4e14c67e"
server_key_version: "encryptionkey@0"
I20260807 18:14:04.133103 19775 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 18:14:04.137370 19795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:04.139101 19775 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 18:14:04.139353 19775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-1/wal
uuid: "aafbbd4e646344f5a4e257344c5f9ddd"
format_stamp: "Formatted at 2026-08-07 18:14:04 on dist-test-slave-39l8"
server_key: "462389bf7fd8ecb5e8a45586ca5f3b68"
server_key_iv: "a221c6cee483b21bf918c92b4e14c67e"
server_key_version: "encryptionkey@0"
I20260807 18:14:04.139613 19775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:04.154678 19775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:04.160153 19775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:04.160472 19775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:04.163970 19775 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:14:04.164103 19775 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:04.164376 19775 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:14:04.164497 19775 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:14:04 dist-test-slave-39l8 krb5kdc[19400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126444, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:04.192703 19807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:04.164483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:39361 (local address 127.17.110.62:34857)
0807 18:14:04.165103 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:04.165111 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:04.166504 (+ 1393us) server_negotiation.cc:407] Connection header received
0807 18:14:04.169085 (+ 2581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:04.169094 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:04.169209 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:04.169365 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:04.172054 (+ 2689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.173233 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.176972 (+ 3739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.177568 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.177725 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:04.184173 (+ 6448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:04.184178 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:04.184182 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:04.184222 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:04.188179 (+ 3957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:04.189316 (+ 1137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:04.189326 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:04.189334 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:04.189539 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:04.190210 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:04.190216 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:04.190220 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:04.190691 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:04.190876 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:04.191928 (+ 1052us) server_negotiation.cc:299] Negotiation successful
0807 18:14:04.192288 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":314,"threads_started":1}
I20260807 18:14:04.194378 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:04.165254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:34857 (local address 127.17.110.2:39361)
0807 18:14:04.166324 (+ 1070us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:04.166374 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:04.168518 (+ 2144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:04.169736 (+ 1218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:04.169747 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:04.170664 (+ 917us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:04.171809 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:04.171851 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.173466 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.173473 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:04.176707 (+ 3234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:04.176756 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.178151 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.178162 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:04.178493 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:04.179578 (+ 1085us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:04.179624 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:04.183797 (+ 4173us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:04.188420 (+ 4623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:04.188435 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:04.188464 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:04.189109 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:04.189696 (+ 587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:04.189704 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:04.189713 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:04.190057 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:04.190858 (+ 801us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:04.190867 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:04.191488 (+ 621us) client_negotiation.cc:769] Sending connection context
0807 18:14:04.191815 (+ 327us) client_negotiation.cc:240] Negotiation successful
0807 18:14:04.192511 (+ 696us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"thread_start_us":485,"threads_started":1}
I20260807 18:14:04.214252 19775 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.2:38765
I20260807 18:14:04.214277 19983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.2:38765 every 8 connection(s)
I20260807 18:14:04.215907 19775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
I20260807 18:14:04.218461 19984 heartbeater.cc:347] Connected to a master server at 127.17.110.62:34857
I20260807 18:14:04.218811 19984 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:04.219800 19984 heartbeater.cc:511] Master 127.17.110.62:34857 requested a full tablet report, sending...
I20260807 18:14:04.220278 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 19775
I20260807 18:14:04.220502 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance
I20260807 18:14:04.221005 19807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:04.203814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:51095 (local address 127.17.110.62:34857)
0807 18:14:04.204065 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:04.204075 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:04.205056 (+ 981us) server_negotiation.cc:407] Connection header received
0807 18:14:04.205323 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:04.205328 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:04.205403 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:04.205566 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:04.207337 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.208774 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.211171 (+ 2397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.211928 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.212095 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:04.214242 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:04.214249 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:04.214252 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:04.214287 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:04.217514 (+ 3227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:04.218454 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:04.218461 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:04.218464 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:04.218631 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:04.219126 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:04.219131 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:04.219135 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:04.219663 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:04.219860 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:04.220489 (+ 629us) server_negotiation.cc:299] Negotiation successful
0807 18:14:04.220781 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 18:14:04.221089 17848 external_mini_cluster.cc:1428] Setting key 6c09a39555f2c69fc28e7face0751142
I20260807 18:14:04.221237 19945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:04.204198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:34857 (local address 127.17.110.2:51095)
0807 18:14:04.204912 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:04.204935 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:04.205158 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:04.205629 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:04.205638 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:04.206510 (+ 872us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:04.207115 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:04.207139 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.208946 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.208954 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:04.210996 (+ 2042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:04.211014 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.212146 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.212151 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:04.212270 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:04.213033 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:04.213055 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:04.214004 (+ 949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:04.217681 (+ 3677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:04.217686 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:04.217690 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:04.218283 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:04.218811 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:04.218822 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:04.218824 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:04.219009 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:04.219835 (+ 826us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:04.219843 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:04.220092 (+ 249us) client_negotiation.cc:769] Sending connection context
0807 18:14:04.220355 (+ 263us) client_negotiation.cc:240] Negotiation successful
0807 18:14:04.220768 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":411,"threads_started":1}
I20260807 18:14:04.222854 19457 ts_manager.cc:194] Registered new tserver with Master: aafbbd4e646344f5a4e257344c5f9ddd (127.17.110.2:38765)
I20260807 18:14:04.224159 19457 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.2@KRBTEST.COM'} at 127.17.110.2:39361
WARNING: no policy specified for kudu/127.17.110.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
I20260807 18:14:04.274549 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.3:0
--local_ip_for_outbound_sockets=127.17.110.3
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:34857
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:39065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:14:04.333854 19765 heartbeater.cc:503] Master 127.17.110.62:34857 was elected leader, sending a full tablet report...
W20260807 18:14:04.644743 19988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:04.645128 19988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:04.645299 19988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:04.652809 19988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:04.652992 19988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:04.653162 19988 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:14:04.653312 19988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.3
I20260807 18:14:04.663836 19988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:39065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.110.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.110.62:34857
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.3
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:04.665280 19988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:04.666833 19988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:04.681033 19998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:04.681094 20001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:04.681016 19997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:04.682586 19988 server_base.cc:1034] running on GCE node
I20260807 18:14:04.683343 19988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:04.684392 19988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:04.685708 19988 hybrid_clock.cc:648] HybridClock initialized: now 1786126444685593 us; error 78 us; skew 500 ppm
I20260807 18:14:04.686090 19988 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:04 dist-test-slave-39l8 krb5kdc[19400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126444, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:04.689903 19988 init.cc:377] Logged in from keytab as kudu/127.17.110.3@KRBTEST.COM (short username kudu)
I20260807 18:14:04.691581 19988 webserver.cc:460] Webserver started at http://127.17.110.3:44491/ using document root <none> and password file <none>
I20260807 18:14:04.692339 19988 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:04.692433 19988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:04.692718 19988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:04.695874 19988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-2/data/instance:
uuid: "8ff251b34c0b429ba9ed605067ceec11"
format_stamp: "Formatted at 2026-08-07 18:14:04 on dist-test-slave-39l8"
server_key: "cc764e0d55ea297ff4ed04d9e6fb89c5"
server_key_iv: "43d580ecde96e5c58cb2b5a48ca153a4"
server_key_version: "encryptionkey@0"
I20260807 18:14:04.696772 19988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance:
uuid: "8ff251b34c0b429ba9ed605067ceec11"
format_stamp: "Formatted at 2026-08-07 18:14:04 on dist-test-slave-39l8"
server_key: "cc764e0d55ea297ff4ed04d9e6fb89c5"
server_key_iv: "43d580ecde96e5c58cb2b5a48ca153a4"
server_key_version: "encryptionkey@0"
I20260807 18:14:04.702780 19988 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 18:14:04.706907 20014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:04.708889 19988 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260807 18:14:04.709065 19988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-2/wal
uuid: "8ff251b34c0b429ba9ed605067ceec11"
format_stamp: "Formatted at 2026-08-07 18:14:04 on dist-test-slave-39l8"
server_key: "cc764e0d55ea297ff4ed04d9e6fb89c5"
server_key_iv: "43d580ecde96e5c58cb2b5a48ca153a4"
server_key_version: "encryptionkey@0"
I20260807 18:14:04.709288 19988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:04.729070 19988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:04.733877 19988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:04.734138 19988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:04.737015 19988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:14:04.737170 19988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:04.737339 19988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:14:04.737411 19988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:14:04 dist-test-slave-39l8 krb5kdc[19400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126444, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:04.766579 20029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:04.737944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:43279 (local address 127.17.110.62:34857)
0807 18:14:04.739190 (+ 1246us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:04.739199 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:04.739831 (+ 632us) server_negotiation.cc:407] Connection header received
0807 18:14:04.742477 (+ 2646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:04.742489 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:04.742602 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:04.742709 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:04.745285 (+ 2576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.746449 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.750010 (+ 3561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.750657 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.750846 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:04.757080 (+ 6234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:04.757093 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:04.757101 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:04.757159 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:04.761603 (+ 4444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:04.762955 (+ 1352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:04.762962 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:04.762967 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:04.763191 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:04.763928 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:04.763937 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:04.763943 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:04.764435 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:04.764620 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:04.765537 (+ 917us) server_negotiation.cc:299] Negotiation successful
0807 18:14:04.765935 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1089,"thread_start_us":975,"threads_started":1}
I20260807 18:14:04.767321 20030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:04.738668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:34857 (local address 127.17.110.3:43279)
0807 18:14:04.739633 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:04.739685 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:04.742045 (+ 2360us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:04.743368 (+ 1323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:04.743388 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:04.744329 (+ 941us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:04.745063 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:04.745085 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.746736 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.746749 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:04.749751 (+ 3002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:04.749778 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.751024 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.751035 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:04.751458 (+ 423us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:04.752513 (+ 1055us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:04.752569 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:04.756631 (+ 4062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:04.761869 (+ 5238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:04.761887 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:04.761918 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:04.762774 (+ 856us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:04.763385 (+ 611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:04.763391 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:04.763395 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:04.763742 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:04.764621 (+ 879us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:04.764633 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:04.765079 (+ 446us) client_negotiation.cc:769] Sending connection context
0807 18:14:04.765346 (+ 267us) client_negotiation.cc:240] Negotiation successful
0807 18:14:04.765916 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"thread_start_us":405,"threads_started":1}
I20260807 18:14:04.793149 19988 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.3:37217
I20260807 18:14:04.793273 20200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.3:37217 every 8 connection(s)
I20260807 18:14:04.794692 19988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
I20260807 18:14:04.797606 20201 heartbeater.cc:347] Connected to a master server at 127.17.110.62:34857
I20260807 18:14:04.797889 20201 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:04.798879 20201 heartbeater.cc:511] Master 127.17.110.62:34857 requested a full tablet report, sending...
I20260807 18:14:04.799571 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 19988
I20260807 18:14:04.799827 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance
I20260807 18:14:04.800364 17848 external_mini_cluster.cc:1428] Setting key e65c64277fc00355dec72ef3ccd1a3ef
I20260807 18:14:04.800863 19457 ts_manager.cc:194] Registered new tserver with Master: 8ff251b34c0b429ba9ed605067ceec11 (127.17.110.3:37217)
I20260807 18:14:04.802093 19457 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.3@KRBTEST.COM'} at 127.17.110.3:43279
I20260807 18:14:04.803905 17848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:14:04.804255 20029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:04.787358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:37493 (local address 127.17.110.62:34857)
0807 18:14:04.787635 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:04.787645 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:04.789023 (+ 1378us) server_negotiation.cc:407] Connection header received
0807 18:14:04.789109 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:04.789115 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:04.789223 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:04.789333 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:04.790902 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.791814 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.793991 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.794441 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.794596 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:04.797436 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:04.797441 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:04.797444 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:04.797481 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:04.801103 (+ 3622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:04.801963 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:04.801970 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:04.801974 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:04.802124 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:04.802562 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:04.802567 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:04.802570 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:04.802986 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:04.803156 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:04.803641 (+ 485us) server_negotiation.cc:299] Negotiation successful
0807 18:14:04.803883 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:14:04.804379 20193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:04.787436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:34857 (local address 127.17.110.3:37493)
0807 18:14:04.788705 (+ 1269us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:04.788727 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:04.788902 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:04.789572 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:04.789578 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:04.790205 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:04.790615 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:04.790633 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.792050 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.792056 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:04.793808 (+ 1752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:04.793826 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.794679 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.794689 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:04.794929 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:04.795718 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:04.795746 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:04.797227 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:04.801318 (+ 4091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:04.801324 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:04.801328 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:04.801833 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:04.802256 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:04.802263 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:04.802271 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:04.802398 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:04.803214 (+ 816us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:04.803218 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:04.803386 (+ 168us) client_negotiation.cc:769] Sending connection context
0807 18:14:04.803547 (+ 161us) client_negotiation.cc:240] Negotiation successful
0807 18:14:04.803902 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1089,"thread_start_us":958,"threads_started":1}
Aug 07 18:14:04 dist-test-slave-39l8 krb5kdc[19400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126444, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 18:14:04 dist-test-slave-39l8 krb5kdc[19400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126444, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:04.861518 20029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:04.844962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45534 (local address 127.17.110.62:34857)
0807 18:14:04.845236 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:04.845243 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:04.845486 (+ 243us) server_negotiation.cc:407] Connection header received
0807 18:14:04.845671 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:04.845676 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:04.845752 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:04.845858 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:04.847604 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.848401 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.849711 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.850152 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.850268 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:04.855237 (+ 4969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:04.855244 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:04.855247 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:04.855295 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:04.858298 (+ 3003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:04.859407 (+ 1109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:04.859412 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:04.859415 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:04.859573 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:04.860059 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:04.860063 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:04.860065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:04.860371 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:04.860610 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:04.860952 (+ 342us) server_negotiation.cc:299] Negotiation successful
0807 18:14:04.861142 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 18:14:04.866465 19457 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45534:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:14:04.874065 19457 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:14:04.909966 20216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:04.888731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:37217 (local address 127.0.0.1:37946)
0807 18:14:04.895704 (+ 6973us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:04.895761 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:04.895988 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:04.898253 (+ 2265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:04.898266 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:04.898335 (+ 69us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:14:04.898963 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:04.898982 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.901404 (+ 2422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.901413 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:04.905564 (+ 4151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:04.905589 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.908708 (+ 3119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.908715 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:04.909062 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:04.909148 (+ 86us) client_negotiation.cc:769] Sending connection context
0807 18:14:04.909313 (+ 165us) client_negotiation.cc:240] Negotiation successful
0807 18:14:04.909548 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6652,"thread_start_us":6499,"threads_started":1}
I20260807 18:14:04.910334 20217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:04.889117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37946 (local address 127.17.110.3:37217)
0807 18:14:04.890531 (+ 1414us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:04.890538 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:04.896798 (+ 6260us) server_negotiation.cc:407] Connection header received
0807 18:14:04.897011 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:04.897023 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:04.897412 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:04.897699 (+ 287us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:14:04.899299 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.901173 (+ 1874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.905819 (+ 4646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.907969 (+ 2150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.908160 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:04.908511 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:04.909767 (+ 1256us) server_negotiation.cc:299] Negotiation successful
0807 18:14:04.909974 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1257,"thread_start_us":1068,"threads_started":1}
I20260807 18:14:04.915058 20089 tablet_service.cc:1467] Processing CreateTablet for tablet ea160eeec79d451ea33df1416a8b7367 (DEFAULT_TABLE table=test-table [id=ac2fc4eca3b0476e99501e317f555441]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:14:04.916697 20089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea160eeec79d451ea33df1416a8b7367. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:04.918185 20224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:04.896486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50322 (local address 127.17.110.2:38765)
0807 18:14:04.903348 (+ 6862us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:04.903361 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:04.909572 (+ 6211us) server_negotiation.cc:407] Connection header received
0807 18:14:04.909696 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:04.909702 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:04.909931 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:04.910136 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:14:04.911208 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.912784 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.916060 (+ 3276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.917063 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.917249 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:04.917475 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:04.917707 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 18:14:04.917883 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6623,"thread_start_us":6407,"threads_started":1}
I20260807 18:14:04.918272 20223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:04.896446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:38765 (local address 127.0.0.1:50322)
0807 18:14:04.909418 (+ 12972us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:04.909436 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:04.909559 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:04.910403 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:04.910410 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:04.910437 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:14:04.910987 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:04.911001 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.913030 (+ 2029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.913037 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:04.915782 (+ 2745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:04.915802 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.917310 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.917316 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:04.917485 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:04.917526 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 18:14:04.917643 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 18:14:04.917802 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12841,"mutex_wait_us":187,"thread_start_us":12658,"threads_started":1}
I20260807 18:14:04.919370 20220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:04.890169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57802 (local address 127.17.110.1:35811)
0807 18:14:04.909119 (+ 18950us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:04.909130 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:04.909172 (+ 42us) server_negotiation.cc:407] Connection header received
0807 18:14:04.909291 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:04.909299 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:04.909597 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:04.909834 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:14:04.911062 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.912747 (+ 1685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.915990 (+ 3243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.917544 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.917703 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:04.917917 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:04.918855 (+ 938us) server_negotiation.cc:299] Negotiation successful
0807 18:14:04.919047 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18703,"thread_start_us":18520,"threads_started":1}
I20260807 18:14:04.919868 20219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:04.889480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:35811 (local address 127.0.0.1:57802)
0807 18:14:04.908230 (+ 18750us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:04.908258 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:04.908437 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:04.910281 (+ 1844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:04.910286 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:04.910312 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:14:04.910802 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:04.910823 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.912958 (+ 2135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.912965 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:04.915789 (+ 2824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:04.915802 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.918090 (+ 2288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.918098 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:04.918297 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:04.918341 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 18:14:04.918431 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 18:14:04.918573 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18529,"mutex_wait_us":158,"thread_start_us":20537,"threads_started":1}
I20260807 18:14:04.922564 19875 tablet_service.cc:1467] Processing CreateTablet for tablet ea160eeec79d451ea33df1416a8b7367 (DEFAULT_TABLE table=test-table [id=ac2fc4eca3b0476e99501e317f555441]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:14:04.923568 19657 tablet_service.cc:1467] Processing CreateTablet for tablet ea160eeec79d451ea33df1416a8b7367 (DEFAULT_TABLE table=test-table [id=ac2fc4eca3b0476e99501e317f555441]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:14:04.924233 19875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea160eeec79d451ea33df1416a8b7367. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:04.925040 19657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea160eeec79d451ea33df1416a8b7367. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:04.931269 20226 tablet_bootstrap.cc:492] T ea160eeec79d451ea33df1416a8b7367 P 8ff251b34c0b429ba9ed605067ceec11: Bootstrap starting.
I20260807 18:14:04.934512 20226 tablet_bootstrap.cc:654] T ea160eeec79d451ea33df1416a8b7367 P 8ff251b34c0b429ba9ed605067ceec11: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:04.936015 20226 log.cc:824] T ea160eeec79d451ea33df1416a8b7367 P 8ff251b34c0b429ba9ed605067ceec11: Log is configured to *not* fsync() on all Append() calls
I20260807 18:14:04.939031 20226 tablet_bootstrap.cc:492] T ea160eeec79d451ea33df1416a8b7367 P 8ff251b34c0b429ba9ed605067ceec11: No bootstrap required, opened a new log
I20260807 18:14:04.939481 20226 ts_tablet_manager.cc:1397] T ea160eeec79d451ea33df1416a8b7367 P 8ff251b34c0b429ba9ed605067ceec11: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260807 18:14:04.940436 20228 tablet_bootstrap.cc:492] T ea160eeec79d451ea33df1416a8b7367 P aafbbd4e646344f5a4e257344c5f9ddd: Bootstrap starting.
I20260807 18:14:04.942459 20229 tablet_bootstrap.cc:492] T ea160eeec79d451ea33df1416a8b7367 P 1131279766ec49aba2aae37aa315c5a7: Bootstrap starting.
I20260807 18:14:04.944496 20228 tablet_bootstrap.cc:654] T ea160eeec79d451ea33df1416a8b7367 P aafbbd4e646344f5a4e257344c5f9ddd: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:04.945932 20229 tablet_bootstrap.cc:654] T ea160eeec79d451ea33df1416a8b7367 P 1131279766ec49aba2aae37aa315c5a7: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:04.945879 20228 log.cc:824] T ea160eeec79d451ea33df1416a8b7367 P aafbbd4e646344f5a4e257344c5f9ddd: Log is configured to *not* fsync() on all Append() calls
I20260807 18:14:04.947333 20229 log.cc:824] T ea160eeec79d451ea33df1416a8b7367 P 1131279766ec49aba2aae37aa315c5a7: Log is configured to *not* fsync() on all Append() calls
I20260807 18:14:04.949524 20228 tablet_bootstrap.cc:492] T ea160eeec79d451ea33df1416a8b7367 P aafbbd4e646344f5a4e257344c5f9ddd: No bootstrap required, opened a new log
I20260807 18:14:04.949849 20228 ts_tablet_manager.cc:1397] T ea160eeec79d451ea33df1416a8b7367 P aafbbd4e646344f5a4e257344c5f9ddd: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260807 18:14:04.952183 20229 tablet_bootstrap.cc:492] T ea160eeec79d451ea33df1416a8b7367 P 1131279766ec49aba2aae37aa315c5a7: No bootstrap required, opened a new log
I20260807 18:14:04.952484 20229 ts_tablet_manager.cc:1397] T ea160eeec79d451ea33df1416a8b7367 P 1131279766ec49aba2aae37aa315c5a7: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260807 18:14:04.952214 20226 raft_consensus.cc:348] T ea160eeec79d451ea33df1416a8b7367 P 8ff251b34c0b429ba9ed605067ceec11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } }
I20260807 18:14:04.952732 20226 raft_consensus.cc:374] T ea160eeec79d451ea33df1416a8b7367 P 8ff251b34c0b429ba9ed605067ceec11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:04.952904 20226 raft_consensus.cc:729] T ea160eeec79d451ea33df1416a8b7367 P 8ff251b34c0b429ba9ed605067ceec11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ff251b34c0b429ba9ed605067ceec11, State: Initialized, Role: FOLLOWER
I20260807 18:14:04.953712 20226 consensus_queue.cc:260] T ea160eeec79d451ea33df1416a8b7367 P 8ff251b34c0b429ba9ed605067ceec11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } }
I20260807 18:14:04.958673 20201 heartbeater.cc:503] Master 127.17.110.62:34857 was elected leader, sending a full tablet report...
I20260807 18:14:04.959692 20226 ts_tablet_manager.cc:1428] T ea160eeec79d451ea33df1416a8b7367 P 8ff251b34c0b429ba9ed605067ceec11: Time spent starting tablet: real 0.019s user 0.014s sys 0.004s
I20260807 18:14:04.962385 20229 raft_consensus.cc:348] T ea160eeec79d451ea33df1416a8b7367 P 1131279766ec49aba2aae37aa315c5a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } }
I20260807 18:14:04.962956 20229 raft_consensus.cc:374] T ea160eeec79d451ea33df1416a8b7367 P 1131279766ec49aba2aae37aa315c5a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:04.963080 20229 raft_consensus.cc:729] T ea160eeec79d451ea33df1416a8b7367 P 1131279766ec49aba2aae37aa315c5a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1131279766ec49aba2aae37aa315c5a7, State: Initialized, Role: FOLLOWER
I20260807 18:14:04.963238 20228 raft_consensus.cc:348] T ea160eeec79d451ea33df1416a8b7367 P aafbbd4e646344f5a4e257344c5f9ddd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } }
I20260807 18:14:04.963680 20228 raft_consensus.cc:374] T ea160eeec79d451ea33df1416a8b7367 P aafbbd4e646344f5a4e257344c5f9ddd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:04.963795 20228 raft_consensus.cc:729] T ea160eeec79d451ea33df1416a8b7367 P aafbbd4e646344f5a4e257344c5f9ddd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aafbbd4e646344f5a4e257344c5f9ddd, State: Initialized, Role: FOLLOWER
I20260807 18:14:04.964025 20229 consensus_queue.cc:260] T ea160eeec79d451ea33df1416a8b7367 P 1131279766ec49aba2aae37aa315c5a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } }
I20260807 18:14:04.964458 20228 consensus_queue.cc:260] T ea160eeec79d451ea33df1416a8b7367 P aafbbd4e646344f5a4e257344c5f9ddd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } }
I20260807 18:14:04.966804 19984 heartbeater.cc:503] Master 127.17.110.62:34857 was elected leader, sending a full tablet report...
I20260807 18:14:04.967505 20228 ts_tablet_manager.cc:1428] T ea160eeec79d451ea33df1416a8b7367 P aafbbd4e646344f5a4e257344c5f9ddd: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
I20260807 18:14:04.967896 20229 ts_tablet_manager.cc:1428] T ea160eeec79d451ea33df1416a8b7367 P 1131279766ec49aba2aae37aa315c5a7: Time spent starting tablet: real 0.015s user 0.008s sys 0.008s
W20260807 18:14:04.968331 19985 tablet.cc:2369] T ea160eeec79d451ea33df1416a8b7367 P aafbbd4e646344f5a4e257344c5f9ddd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:14:04.981201 20234 raft_consensus.cc:482] T ea160eeec79d451ea33df1416a8b7367 P 1131279766ec49aba2aae37aa315c5a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:14:04.981594 20234 raft_consensus.cc:504] T ea160eeec79d451ea33df1416a8b7367 P 1131279766ec49aba2aae37aa315c5a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } }
I20260807 18:14:04.983696 20234 leader_election.cc:288] T ea160eeec79d451ea33df1416a8b7367 P 1131279766ec49aba2aae37aa315c5a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ff251b34c0b429ba9ed605067ceec11 (127.17.110.3:37217), aafbbd4e646344f5a4e257344c5f9ddd (127.17.110.2:38765)
I20260807 18:14:04.992034 20217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:04.984241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:39565 (local address 127.17.110.3:37217)
0807 18:14:04.984461 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:04.984466 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:04.985193 (+ 727us) server_negotiation.cc:407] Connection header received
0807 18:14:04.985411 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:04.985419 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:04.985506 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:04.985620 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:14:04.986535 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.987443 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.989806 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.990964 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.991130 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:04.991220 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:04.991605 (+ 385us) server_negotiation.cc:299] Negotiation successful
0807 18:14:04.991761 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:14:04.992087 20236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:04.984122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:37217 (local address 127.17.110.1:39565)
0807 18:14:04.985057 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:04.985086 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:04.985242 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:04.985845 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:04.985849 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:04.985875 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:14:04.986330 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:04.986346 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.987633 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.987638 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:04.989628 (+ 1990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:04.989643 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.991306 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.991311 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:04.991431 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:04.991457 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:14:04.991541 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 18:14:04.991644 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"thread_start_us":508,"threads_started":1}
I20260807 18:14:04.992703 20224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:04.985531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:46161 (local address 127.17.110.2:38765)
0807 18:14:04.985741 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:04.985747 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:04.986081 (+ 334us) server_negotiation.cc:407] Connection header received
0807 18:14:04.986277 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:04.986283 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:04.986363 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:04.986481 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:14:04.987289 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.988332 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.990495 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.991634 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.991818 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:04.991893 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:04.992356 (+ 463us) server_negotiation.cc:299] Negotiation successful
0807 18:14:04.992503 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 18:14:04.992785 20238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:04.985241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:38765 (local address 127.17.110.1:46161)
0807 18:14:04.985961 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:04.985980 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:04.986093 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:04.986817 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:04.986821 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:04.986841 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:14:04.987128 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:04.987135 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.988472 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.988476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:04.990310 (+ 1834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:04.990326 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:04.991886 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:04.991895 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:04.992087 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:04.992130 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 18:14:04.992301 (+ 171us) client_negotiation.cc:240] Negotiation successful
0807 18:14:04.992456 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":341,"threads_started":1}
I20260807 18:14:04.993292 20119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea160eeec79d451ea33df1416a8b7367" candidate_uuid: "1131279766ec49aba2aae37aa315c5a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ff251b34c0b429ba9ed605067ceec11" is_pre_election: true
I20260807 18:14:04.994004 20119 raft_consensus.cc:2454] T ea160eeec79d451ea33df1416a8b7367 P 8ff251b34c0b429ba9ed605067ceec11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1131279766ec49aba2aae37aa315c5a7 in term 0.
I20260807 18:14:04.993839 19902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea160eeec79d451ea33df1416a8b7367" candidate_uuid: "1131279766ec49aba2aae37aa315c5a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" is_pre_election: true
I20260807 18:14:04.994518 19902 raft_consensus.cc:2454] T ea160eeec79d451ea33df1416a8b7367 P aafbbd4e646344f5a4e257344c5f9ddd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1131279766ec49aba2aae37aa315c5a7 in term 0.
I20260807 18:14:04.994992 19583 leader_election.cc:302] T ea160eeec79d451ea33df1416a8b7367 P 1131279766ec49aba2aae37aa315c5a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1131279766ec49aba2aae37aa315c5a7, 8ff251b34c0b429ba9ed605067ceec11; no voters:
I20260807 18:14:04.995488 20234 raft_consensus.cc:2781] T ea160eeec79d451ea33df1416a8b7367 P 1131279766ec49aba2aae37aa315c5a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:14:04.995657 20234 raft_consensus.cc:482] T ea160eeec79d451ea33df1416a8b7367 P 1131279766ec49aba2aae37aa315c5a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:14:04.995779 20234 raft_consensus.cc:3037] T ea160eeec79d451ea33df1416a8b7367 P 1131279766ec49aba2aae37aa315c5a7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:04.998355 20234 raft_consensus.cc:504] T ea160eeec79d451ea33df1416a8b7367 P 1131279766ec49aba2aae37aa315c5a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } }
I20260807 18:14:04.999101 20234 leader_election.cc:288] T ea160eeec79d451ea33df1416a8b7367 P 1131279766ec49aba2aae37aa315c5a7 [CANDIDATE]: Term 1 election: Requested vote from peers 8ff251b34c0b429ba9ed605067ceec11 (127.17.110.3:37217), aafbbd4e646344f5a4e257344c5f9ddd (127.17.110.2:38765)
I20260807 18:14:04.999624 20119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea160eeec79d451ea33df1416a8b7367" candidate_uuid: "1131279766ec49aba2aae37aa315c5a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ff251b34c0b429ba9ed605067ceec11"
I20260807 18:14:04.999688 19902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea160eeec79d451ea33df1416a8b7367" candidate_uuid: "1131279766ec49aba2aae37aa315c5a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aafbbd4e646344f5a4e257344c5f9ddd"
I20260807 18:14:04.999871 20119 raft_consensus.cc:3037] T ea160eeec79d451ea33df1416a8b7367 P 8ff251b34c0b429ba9ed605067ceec11 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:04.999922 19902 raft_consensus.cc:3037] T ea160eeec79d451ea33df1416a8b7367 P aafbbd4e646344f5a4e257344c5f9ddd [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:05.002364 19902 raft_consensus.cc:2454] T ea160eeec79d451ea33df1416a8b7367 P aafbbd4e646344f5a4e257344c5f9ddd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1131279766ec49aba2aae37aa315c5a7 in term 1.
I20260807 18:14:05.002534 20119 raft_consensus.cc:2454] T ea160eeec79d451ea33df1416a8b7367 P 8ff251b34c0b429ba9ed605067ceec11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1131279766ec49aba2aae37aa315c5a7 in term 1.
I20260807 18:14:05.002934 19583 leader_election.cc:302] T ea160eeec79d451ea33df1416a8b7367 P 1131279766ec49aba2aae37aa315c5a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1131279766ec49aba2aae37aa315c5a7, aafbbd4e646344f5a4e257344c5f9ddd; no voters:
I20260807 18:14:05.003329 20234 raft_consensus.cc:2781] T ea160eeec79d451ea33df1416a8b7367 P 1131279766ec49aba2aae37aa315c5a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:14:05.003857 20234 raft_consensus.cc:686] T ea160eeec79d451ea33df1416a8b7367 P 1131279766ec49aba2aae37aa315c5a7 [term 1 LEADER]: Becoming Leader. State: Replica: 1131279766ec49aba2aae37aa315c5a7, State: Running, Role: LEADER
I20260807 18:14:05.004391 20234 consensus_queue.cc:237] T ea160eeec79d451ea33df1416a8b7367 P 1131279766ec49aba2aae37aa315c5a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } }
I20260807 18:14:05.010172 19458 catalog_manager.cc:5495] T ea160eeec79d451ea33df1416a8b7367 P 1131279766ec49aba2aae37aa315c5a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1131279766ec49aba2aae37aa315c5a7 (127.17.110.1). New cstate: current_term: 1 leader_uuid: "1131279766ec49aba2aae37aa315c5a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } health_report { overall_health: HEALTHY } } }
I20260807 18:14:05.036571 20220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:05.029999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57818 (local address 127.17.110.1:35811)
0807 18:14:05.030277 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:05.030285 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:05.030356 (+ 71us) server_negotiation.cc:407] Connection header received
0807 18:14:05.030665 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:05.030670 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:05.030828 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:05.030958 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:14:05.031332 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:05.032053 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:05.033793 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:05.034401 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:05.034525 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:05.035267 (+ 742us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:14:05.035850 (+ 583us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:14:05.035985 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:05.036154 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 18:14:05.036232 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260807 18:14:05.046692 20202 tablet.cc:2369] T ea160eeec79d451ea33df1416a8b7367 P 8ff251b34c0b429ba9ed605067ceec11: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:14:05.047713 19902 raft_consensus.cc:1261] T ea160eeec79d451ea33df1416a8b7367 P aafbbd4e646344f5a4e257344c5f9ddd [term 1 FOLLOWER]: Refusing update from remote peer 1131279766ec49aba2aae37aa315c5a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:14:05.048521 20119 raft_consensus.cc:1261] T ea160eeec79d451ea33df1416a8b7367 P 8ff251b34c0b429ba9ed605067ceec11 [term 1 FOLLOWER]: Refusing update from remote peer 1131279766ec49aba2aae37aa315c5a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:14:05.048966 20241 consensus_queue.cc:1035] T ea160eeec79d451ea33df1416a8b7367 P 1131279766ec49aba2aae37aa315c5a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:14:05.049362 20234 consensus_queue.cc:1035] T ea160eeec79d451ea33df1416a8b7367 P 1131279766ec49aba2aae37aa315c5a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:14:05.084602 19458 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:14:05.085007 19458 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:14:05.089493 19458 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ca311ba787804c1b9e854c216d9e7289: Sending DeleteTablet for 3 replicas of tablet ea160eeec79d451ea33df1416a8b7367
I20260807 18:14:05.091188 19657 tablet_service.cc:1514] Processing DeleteTablet for tablet ea160eeec79d451ea33df1416a8b7367 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:14:05 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:57802
I20260807 18:14:05.091529 20089 tablet_service.cc:1514] Processing DeleteTablet for tablet ea160eeec79d451ea33df1416a8b7367 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:14:05 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:37946
I20260807 18:14:05.091499 19875 tablet_service.cc:1514] Processing DeleteTablet for tablet ea160eeec79d451ea33df1416a8b7367 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:14:05 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:50322
I20260807 18:14:05.093626 19458 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45534:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:14:05.095966 19458 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:14:05.110625 19874 tablet_service.cc:1467] Processing CreateTablet for tablet b726a125754b4d11917ddc838d3b7660 (DEFAULT_TABLE table=test-table [id=28ad03bcea8d4b419cdddcb506f105d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:14:05.110625 20088 tablet_service.cc:1467] Processing CreateTablet for tablet b726a125754b4d11917ddc838d3b7660 (DEFAULT_TABLE table=test-table [id=28ad03bcea8d4b419cdddcb506f105d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:14:05.111408 20088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b726a125754b4d11917ddc838d3b7660. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:05.111446 19874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b726a125754b4d11917ddc838d3b7660. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:05.115916 19656 tablet_service.cc:1467] Processing CreateTablet for tablet b726a125754b4d11917ddc838d3b7660 (DEFAULT_TABLE table=test-table [id=28ad03bcea8d4b419cdddcb506f105d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:14:05.116617 19656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b726a125754b4d11917ddc838d3b7660. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:05.118482 20261 tablet_replica.cc:330] T ea160eeec79d451ea33df1416a8b7367 P aafbbd4e646344f5a4e257344c5f9ddd: stopping tablet replica
I20260807 18:14:05.119639 20260 tablet_replica.cc:330] T ea160eeec79d451ea33df1416a8b7367 P 8ff251b34c0b429ba9ed605067ceec11: stopping tablet replica
I20260807 18:14:05.120659 20260 raft_consensus.cc:2229] T ea160eeec79d451ea33df1416a8b7367 P 8ff251b34c0b429ba9ed605067ceec11 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:14:05.121407 20261 raft_consensus.cc:2229] T ea160eeec79d451ea33df1416a8b7367 P aafbbd4e646344f5a4e257344c5f9ddd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:14:05.122244 20259 tablet_replica.cc:330] T ea160eeec79d451ea33df1416a8b7367 P 1131279766ec49aba2aae37aa315c5a7: stopping tablet replica
I20260807 18:14:05.122759 20259 raft_consensus.cc:2229] T ea160eeec79d451ea33df1416a8b7367 P 1131279766ec49aba2aae37aa315c5a7 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:14:05.123227 20226 tablet_bootstrap.cc:492] T b726a125754b4d11917ddc838d3b7660 P 8ff251b34c0b429ba9ed605067ceec11: Bootstrap starting.
I20260807 18:14:05.123699 20259 raft_consensus.cc:2258] T ea160eeec79d451ea33df1416a8b7367 P 1131279766ec49aba2aae37aa315c5a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:14:05.124626 20261 raft_consensus.cc:2258] T ea160eeec79d451ea33df1416a8b7367 P aafbbd4e646344f5a4e257344c5f9ddd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:14:05.125198 20226 tablet_bootstrap.cc:654] T b726a125754b4d11917ddc838d3b7660 P 8ff251b34c0b429ba9ed605067ceec11: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:05.122575 20228 tablet_bootstrap.cc:492] T b726a125754b4d11917ddc838d3b7660 P aafbbd4e646344f5a4e257344c5f9ddd: Bootstrap starting.
I20260807 18:14:05.127688 20259 ts_tablet_manager.cc:1905] T ea160eeec79d451ea33df1416a8b7367 P 1131279766ec49aba2aae37aa315c5a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:14:05.128038 20228 tablet_bootstrap.cc:654] T b726a125754b4d11917ddc838d3b7660 P aafbbd4e646344f5a4e257344c5f9ddd: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:05.129231 20261 ts_tablet_manager.cc:1905] T ea160eeec79d451ea33df1416a8b7367 P aafbbd4e646344f5a4e257344c5f9ddd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:14:05.128932 20229 tablet_bootstrap.cc:492] T b726a125754b4d11917ddc838d3b7660 P 1131279766ec49aba2aae37aa315c5a7: Bootstrap starting.
I20260807 18:14:05.131232 20229 tablet_bootstrap.cc:654] T b726a125754b4d11917ddc838d3b7660 P 1131279766ec49aba2aae37aa315c5a7: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:05.134521 20260 raft_consensus.cc:2258] T ea160eeec79d451ea33df1416a8b7367 P 8ff251b34c0b429ba9ed605067ceec11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:14:05.143718 20260 ts_tablet_manager.cc:1905] T ea160eeec79d451ea33df1416a8b7367 P 8ff251b34c0b429ba9ed605067ceec11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:14:05.144681 20229 tablet_bootstrap.cc:492] T b726a125754b4d11917ddc838d3b7660 P 1131279766ec49aba2aae37aa315c5a7: No bootstrap required, opened a new log
I20260807 18:14:05.144860 20229 ts_tablet_manager.cc:1397] T b726a125754b4d11917ddc838d3b7660 P 1131279766ec49aba2aae37aa315c5a7: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.001s
I20260807 18:14:05.144819 20261 ts_tablet_manager.cc:1918] T ea160eeec79d451ea33df1416a8b7367 P aafbbd4e646344f5a4e257344c5f9ddd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:14:05.145001 20261 log.cc:1196] T ea160eeec79d451ea33df1416a8b7367 P aafbbd4e646344f5a4e257344c5f9ddd: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-1/wal/wals/ea160eeec79d451ea33df1416a8b7367
I20260807 18:14:05.145678 20261 ts_tablet_manager.cc:1939] T ea160eeec79d451ea33df1416a8b7367 P aafbbd4e646344f5a4e257344c5f9ddd: Deleting consensus metadata
I20260807 18:14:05.145900 20229 raft_consensus.cc:348] T b726a125754b4d11917ddc838d3b7660 P 1131279766ec49aba2aae37aa315c5a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } }
I20260807 18:14:05.146126 20229 raft_consensus.cc:374] T b726a125754b4d11917ddc838d3b7660 P 1131279766ec49aba2aae37aa315c5a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:05.146226 20229 raft_consensus.cc:729] T b726a125754b4d11917ddc838d3b7660 P 1131279766ec49aba2aae37aa315c5a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1131279766ec49aba2aae37aa315c5a7, State: Initialized, Role: FOLLOWER
I20260807 18:14:05.146494 20229 consensus_queue.cc:260] T b726a125754b4d11917ddc838d3b7660 P 1131279766ec49aba2aae37aa315c5a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } }
I20260807 18:14:05.147358 20226 tablet_bootstrap.cc:492] T b726a125754b4d11917ddc838d3b7660 P 8ff251b34c0b429ba9ed605067ceec11: No bootstrap required, opened a new log
I20260807 18:14:05.147537 20226 ts_tablet_manager.cc:1397] T b726a125754b4d11917ddc838d3b7660 P 8ff251b34c0b429ba9ed605067ceec11: Time spent bootstrapping tablet: real 0.026s user 0.009s sys 0.000s
I20260807 18:14:05.147672 20228 tablet_bootstrap.cc:492] T b726a125754b4d11917ddc838d3b7660 P aafbbd4e646344f5a4e257344c5f9ddd: No bootstrap required, opened a new log
I20260807 18:14:05.147833 20228 ts_tablet_manager.cc:1397] T b726a125754b4d11917ddc838d3b7660 P aafbbd4e646344f5a4e257344c5f9ddd: Time spent bootstrapping tablet: real 0.025s user 0.007s sys 0.005s
I20260807 18:14:05.147886 19440 catalog_manager.cc:4841] TS aafbbd4e646344f5a4e257344c5f9ddd (127.17.110.2:38765): tablet ea160eeec79d451ea33df1416a8b7367 (table test-table [id=ac2fc4eca3b0476e99501e317f555441]) successfully deleted
I20260807 18:14:05.148458 20226 raft_consensus.cc:348] T b726a125754b4d11917ddc838d3b7660 P 8ff251b34c0b429ba9ed605067ceec11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } }
I20260807 18:14:05.148725 20226 raft_consensus.cc:374] T b726a125754b4d11917ddc838d3b7660 P 8ff251b34c0b429ba9ed605067ceec11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:05.148622 20228 raft_consensus.cc:348] T b726a125754b4d11917ddc838d3b7660 P aafbbd4e646344f5a4e257344c5f9ddd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } }
I20260807 18:14:05.148838 20228 raft_consensus.cc:374] T b726a125754b4d11917ddc838d3b7660 P aafbbd4e646344f5a4e257344c5f9ddd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:05.148828 20226 raft_consensus.cc:729] T b726a125754b4d11917ddc838d3b7660 P 8ff251b34c0b429ba9ed605067ceec11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ff251b34c0b429ba9ed605067ceec11, State: Initialized, Role: FOLLOWER
I20260807 18:14:05.148939 20228 raft_consensus.cc:729] T b726a125754b4d11917ddc838d3b7660 P aafbbd4e646344f5a4e257344c5f9ddd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aafbbd4e646344f5a4e257344c5f9ddd, State: Initialized, Role: FOLLOWER
I20260807 18:14:05.149078 20229 ts_tablet_manager.cc:1428] T b726a125754b4d11917ddc838d3b7660 P 1131279766ec49aba2aae37aa315c5a7: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260807 18:14:05.149175 20226 consensus_queue.cc:260] T b726a125754b4d11917ddc838d3b7660 P 8ff251b34c0b429ba9ed605067ceec11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } }
I20260807 18:14:05.149262 20228 consensus_queue.cc:260] T b726a125754b4d11917ddc838d3b7660 P aafbbd4e646344f5a4e257344c5f9ddd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } }
I20260807 18:14:05.149838 20228 ts_tablet_manager.cc:1428] T b726a125754b4d11917ddc838d3b7660 P aafbbd4e646344f5a4e257344c5f9ddd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 18:14:05.149909 20259 ts_tablet_manager.cc:1918] T ea160eeec79d451ea33df1416a8b7367 P 1131279766ec49aba2aae37aa315c5a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:14:05.150098 20259 log.cc:1196] T ea160eeec79d451ea33df1416a8b7367 P 1131279766ec49aba2aae37aa315c5a7: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-0/wal/wals/ea160eeec79d451ea33df1416a8b7367
I20260807 18:14:05.150800 20259 ts_tablet_manager.cc:1939] T ea160eeec79d451ea33df1416a8b7367 P 1131279766ec49aba2aae37aa315c5a7: Deleting consensus metadata
I20260807 18:14:05.152809 19443 catalog_manager.cc:4841] TS 1131279766ec49aba2aae37aa315c5a7 (127.17.110.1:35811): tablet ea160eeec79d451ea33df1416a8b7367 (table test-table [id=ac2fc4eca3b0476e99501e317f555441]) successfully deleted
I20260807 18:14:05.153577 20226 ts_tablet_manager.cc:1428] T b726a125754b4d11917ddc838d3b7660 P 8ff251b34c0b429ba9ed605067ceec11: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260807 18:14:05.162370 20260 ts_tablet_manager.cc:1918] T ea160eeec79d451ea33df1416a8b7367 P 8ff251b34c0b429ba9ed605067ceec11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:14:05.162662 20260 log.cc:1196] T ea160eeec79d451ea33df1416a8b7367 P 8ff251b34c0b429ba9ed605067ceec11: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-2/wal/wals/ea160eeec79d451ea33df1416a8b7367
I20260807 18:14:05.163398 20260 ts_tablet_manager.cc:1939] T ea160eeec79d451ea33df1416a8b7367 P 8ff251b34c0b429ba9ed605067ceec11: Deleting consensus metadata
I20260807 18:14:05.165755 19440 catalog_manager.cc:4841] TS 8ff251b34c0b429ba9ed605067ceec11 (127.17.110.3:37217): tablet ea160eeec79d451ea33df1416a8b7367 (table test-table [id=ac2fc4eca3b0476e99501e317f555441]) successfully deleted
I20260807 18:14:05.269199 20252 raft_consensus.cc:482] T b726a125754b4d11917ddc838d3b7660 P 1131279766ec49aba2aae37aa315c5a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:14:05.269541 20252 raft_consensus.cc:504] T b726a125754b4d11917ddc838d3b7660 P 1131279766ec49aba2aae37aa315c5a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } }
I20260807 18:14:05.270591 20252 leader_election.cc:288] T b726a125754b4d11917ddc838d3b7660 P 1131279766ec49aba2aae37aa315c5a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aafbbd4e646344f5a4e257344c5f9ddd (127.17.110.2:38765), 8ff251b34c0b429ba9ed605067ceec11 (127.17.110.3:37217)
I20260807 18:14:05.271391 20119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b726a125754b4d11917ddc838d3b7660" candidate_uuid: "1131279766ec49aba2aae37aa315c5a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ff251b34c0b429ba9ed605067ceec11" is_pre_election: true
I20260807 18:14:05.271374 19902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b726a125754b4d11917ddc838d3b7660" candidate_uuid: "1131279766ec49aba2aae37aa315c5a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" is_pre_election: true
I20260807 18:14:05.271665 19902 raft_consensus.cc:2454] T b726a125754b4d11917ddc838d3b7660 P aafbbd4e646344f5a4e257344c5f9ddd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1131279766ec49aba2aae37aa315c5a7 in term 0.
I20260807 18:14:05.271665 20119 raft_consensus.cc:2454] T b726a125754b4d11917ddc838d3b7660 P 8ff251b34c0b429ba9ed605067ceec11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1131279766ec49aba2aae37aa315c5a7 in term 0.
I20260807 18:14:05.272241 19583 leader_election.cc:302] T b726a125754b4d11917ddc838d3b7660 P 1131279766ec49aba2aae37aa315c5a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1131279766ec49aba2aae37aa315c5a7, aafbbd4e646344f5a4e257344c5f9ddd; no voters:
I20260807 18:14:05.272640 20252 raft_consensus.cc:2781] T b726a125754b4d11917ddc838d3b7660 P 1131279766ec49aba2aae37aa315c5a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:14:05.272799 20252 raft_consensus.cc:482] T b726a125754b4d11917ddc838d3b7660 P 1131279766ec49aba2aae37aa315c5a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:14:05.272946 20252 raft_consensus.cc:3037] T b726a125754b4d11917ddc838d3b7660 P 1131279766ec49aba2aae37aa315c5a7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:05.275727 20252 raft_consensus.cc:504] T b726a125754b4d11917ddc838d3b7660 P 1131279766ec49aba2aae37aa315c5a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } }
I20260807 18:14:05.276556 20252 leader_election.cc:288] T b726a125754b4d11917ddc838d3b7660 P 1131279766ec49aba2aae37aa315c5a7 [CANDIDATE]: Term 1 election: Requested vote from peers aafbbd4e646344f5a4e257344c5f9ddd (127.17.110.2:38765), 8ff251b34c0b429ba9ed605067ceec11 (127.17.110.3:37217)
I20260807 18:14:05.277272 20119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b726a125754b4d11917ddc838d3b7660" candidate_uuid: "1131279766ec49aba2aae37aa315c5a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ff251b34c0b429ba9ed605067ceec11"
I20260807 18:14:05.277248 19902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b726a125754b4d11917ddc838d3b7660" candidate_uuid: "1131279766ec49aba2aae37aa315c5a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aafbbd4e646344f5a4e257344c5f9ddd"
I20260807 18:14:05.277539 20119 raft_consensus.cc:3037] T b726a125754b4d11917ddc838d3b7660 P 8ff251b34c0b429ba9ed605067ceec11 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:05.277562 19902 raft_consensus.cc:3037] T b726a125754b4d11917ddc838d3b7660 P aafbbd4e646344f5a4e257344c5f9ddd [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:05.280946 20119 raft_consensus.cc:2454] T b726a125754b4d11917ddc838d3b7660 P 8ff251b34c0b429ba9ed605067ceec11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1131279766ec49aba2aae37aa315c5a7 in term 1.
I20260807 18:14:05.280946 19902 raft_consensus.cc:2454] T b726a125754b4d11917ddc838d3b7660 P aafbbd4e646344f5a4e257344c5f9ddd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1131279766ec49aba2aae37aa315c5a7 in term 1.
I20260807 18:14:05.281478 19583 leader_election.cc:302] T b726a125754b4d11917ddc838d3b7660 P 1131279766ec49aba2aae37aa315c5a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1131279766ec49aba2aae37aa315c5a7, 8ff251b34c0b429ba9ed605067ceec11; no voters:
I20260807 18:14:05.281868 20252 raft_consensus.cc:2781] T b726a125754b4d11917ddc838d3b7660 P 1131279766ec49aba2aae37aa315c5a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:14:05.282145 20252 raft_consensus.cc:686] T b726a125754b4d11917ddc838d3b7660 P 1131279766ec49aba2aae37aa315c5a7 [term 1 LEADER]: Becoming Leader. State: Replica: 1131279766ec49aba2aae37aa315c5a7, State: Running, Role: LEADER
I20260807 18:14:05.282487 20252 consensus_queue.cc:237] T b726a125754b4d11917ddc838d3b7660 P 1131279766ec49aba2aae37aa315c5a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } }
I20260807 18:14:05.285785 19458 catalog_manager.cc:5495] T b726a125754b4d11917ddc838d3b7660 P 1131279766ec49aba2aae37aa315c5a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1131279766ec49aba2aae37aa315c5a7 (127.17.110.1). New cstate: current_term: 1 leader_uuid: "1131279766ec49aba2aae37aa315c5a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } health_report { overall_health: UNKNOWN } } }
W20260807 18:14:05.324766 19769 tablet.cc:2369] T b726a125754b4d11917ddc838d3b7660 P 1131279766ec49aba2aae37aa315c5a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 18:14:05 dist-test-slave-39l8 krb5kdc[19400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126441, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:05.374421 20029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:05.357654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45536 (local address 127.17.110.62:34857)
0807 18:14:05.357865 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:05.357873 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:05.358907 (+ 1034us) server_negotiation.cc:407] Connection header received
0807 18:14:05.359140 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:05.359145 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:05.359239 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:05.359349 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:05.360729 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:05.361487 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:05.362876 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:05.363334 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:05.363474 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:05.367432 (+ 3958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:05.367438 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:05.367444 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:05.367487 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:05.370995 (+ 3508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:05.371945 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:05.371957 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:05.371960 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:05.372115 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:05.372628 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:05.372642 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:05.372644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:05.372945 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:05.373126 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:05.373656 (+ 530us) server_negotiation.cc:299] Negotiation successful
0807 18:14:05.374054 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":149,"server-negotiator.queue_time_us":67}
I20260807 18:14:05.374449 20275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:05.357777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:34857 (local address 127.0.0.1:45536)
0807 18:14:05.358773 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:05.358792 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:05.358938 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:05.359378 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:05.359382 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:05.360259 (+ 877us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:05.360567 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:05.360578 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:05.361625 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:05.361634 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:05.362669 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:05.362682 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:05.363544 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:05.363556 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:05.363715 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:05.364533 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:05.364551 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:05.367195 (+ 2644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:05.371200 (+ 4005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:05.371221 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:05.371231 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:05.371772 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:05.372250 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:05.372257 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:05.372262 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:05.372488 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:05.373126 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:05.373138 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:05.373337 (+ 199us) client_negotiation.cc:769] Sending connection context
0807 18:14:05.373524 (+ 187us) client_negotiation.cc:240] Negotiation successful
0807 18:14:05.373888 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":721,"thread_start_us":564,"threads_started":1}
I20260807 18:14:05.382206 19458 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:45536:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 18:14:05.384013 19458 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:14:05.393297 19656 tablet_service.cc:1467] Processing CreateTablet for tablet 42afdb33548544acb0a18f8373a6634b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=11944117edc749cfb9f00e108f0de15b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:14:05.393601 20088 tablet_service.cc:1467] Processing CreateTablet for tablet 42afdb33548544acb0a18f8373a6634b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=11944117edc749cfb9f00e108f0de15b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:14:05.393852 19875 tablet_service.cc:1467] Processing CreateTablet for tablet 42afdb33548544acb0a18f8373a6634b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=11944117edc749cfb9f00e108f0de15b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:14:05.394133 19656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42afdb33548544acb0a18f8373a6634b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:05.394200 20088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42afdb33548544acb0a18f8373a6634b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:05.394441 19875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42afdb33548544acb0a18f8373a6634b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:05.403585 20226 tablet_bootstrap.cc:492] T 42afdb33548544acb0a18f8373a6634b P 8ff251b34c0b429ba9ed605067ceec11: Bootstrap starting.
I20260807 18:14:05.404367 20228 tablet_bootstrap.cc:492] T 42afdb33548544acb0a18f8373a6634b P aafbbd4e646344f5a4e257344c5f9ddd: Bootstrap starting.
I20260807 18:14:05.405608 20226 tablet_bootstrap.cc:654] T 42afdb33548544acb0a18f8373a6634b P 8ff251b34c0b429ba9ed605067ceec11: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:05.406388 20228 tablet_bootstrap.cc:654] T 42afdb33548544acb0a18f8373a6634b P aafbbd4e646344f5a4e257344c5f9ddd: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:05.408353 20226 tablet_bootstrap.cc:492] T 42afdb33548544acb0a18f8373a6634b P 8ff251b34c0b429ba9ed605067ceec11: No bootstrap required, opened a new log
I20260807 18:14:05.408555 20226 ts_tablet_manager.cc:1397] T 42afdb33548544acb0a18f8373a6634b P 8ff251b34c0b429ba9ed605067ceec11: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 18:14:05.409301 20228 tablet_bootstrap.cc:492] T 42afdb33548544acb0a18f8373a6634b P aafbbd4e646344f5a4e257344c5f9ddd: No bootstrap required, opened a new log
I20260807 18:14:05.409502 20228 ts_tablet_manager.cc:1397] T 42afdb33548544acb0a18f8373a6634b P aafbbd4e646344f5a4e257344c5f9ddd: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260807 18:14:05.409392 20226 raft_consensus.cc:348] T 42afdb33548544acb0a18f8373a6634b P 8ff251b34c0b429ba9ed605067ceec11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } }
I20260807 18:14:05.409670 20226 raft_consensus.cc:374] T 42afdb33548544acb0a18f8373a6634b P 8ff251b34c0b429ba9ed605067ceec11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:05.409762 20226 raft_consensus.cc:729] T 42afdb33548544acb0a18f8373a6634b P 8ff251b34c0b429ba9ed605067ceec11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ff251b34c0b429ba9ed605067ceec11, State: Initialized, Role: FOLLOWER
I20260807 18:14:05.410044 20226 consensus_queue.cc:260] T 42afdb33548544acb0a18f8373a6634b P 8ff251b34c0b429ba9ed605067ceec11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } }
I20260807 18:14:05.410590 20233 tablet_replica.cc:439] T 42afdb33548544acb0a18f8373a6634b P 8ff251b34c0b429ba9ed605067ceec11: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } } }
I20260807 18:14:05.411037 20233 tablet_replica.cc:442] T 42afdb33548544acb0a18f8373a6634b P 8ff251b34c0b429ba9ed605067ceec11: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:14:05.410997 20228 raft_consensus.cc:348] T 42afdb33548544acb0a18f8373a6634b P aafbbd4e646344f5a4e257344c5f9ddd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } }
I20260807 18:14:05.411278 20228 raft_consensus.cc:374] T 42afdb33548544acb0a18f8373a6634b P aafbbd4e646344f5a4e257344c5f9ddd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:05.411365 20228 raft_consensus.cc:729] T 42afdb33548544acb0a18f8373a6634b P aafbbd4e646344f5a4e257344c5f9ddd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aafbbd4e646344f5a4e257344c5f9ddd, State: Initialized, Role: FOLLOWER
I20260807 18:14:05.411644 20228 consensus_queue.cc:260] T 42afdb33548544acb0a18f8373a6634b P aafbbd4e646344f5a4e257344c5f9ddd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } }
I20260807 18:14:05.412223 20226 ts_tablet_manager.cc:1428] T 42afdb33548544acb0a18f8373a6634b P 8ff251b34c0b429ba9ed605067ceec11: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260807 18:14:05.412139 20235 tablet_replica.cc:439] T 42afdb33548544acb0a18f8373a6634b P aafbbd4e646344f5a4e257344c5f9ddd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } } }
I20260807 18:14:05.412540 20235 tablet_replica.cc:442] T 42afdb33548544acb0a18f8373a6634b P aafbbd4e646344f5a4e257344c5f9ddd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:14:05.413518 20229 tablet_bootstrap.cc:492] T 42afdb33548544acb0a18f8373a6634b P 1131279766ec49aba2aae37aa315c5a7: Bootstrap starting.
I20260807 18:14:05.415462 20229 tablet_bootstrap.cc:654] T 42afdb33548544acb0a18f8373a6634b P 1131279766ec49aba2aae37aa315c5a7: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:05.416378 20228 ts_tablet_manager.cc:1428] T 42afdb33548544acb0a18f8373a6634b P aafbbd4e646344f5a4e257344c5f9ddd: Time spent starting tablet: real 0.007s user 0.002s sys 0.001s
I20260807 18:14:05.419075 20229 tablet_bootstrap.cc:492] T 42afdb33548544acb0a18f8373a6634b P 1131279766ec49aba2aae37aa315c5a7: No bootstrap required, opened a new log
I20260807 18:14:05.420449 20229 ts_tablet_manager.cc:1397] T 42afdb33548544acb0a18f8373a6634b P 1131279766ec49aba2aae37aa315c5a7: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.001s
I20260807 18:14:05.421476 20229 raft_consensus.cc:348] T 42afdb33548544acb0a18f8373a6634b P 1131279766ec49aba2aae37aa315c5a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } }
I20260807 18:14:05.421787 20229 raft_consensus.cc:374] T 42afdb33548544acb0a18f8373a6634b P 1131279766ec49aba2aae37aa315c5a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:05.421906 20229 raft_consensus.cc:729] T 42afdb33548544acb0a18f8373a6634b P 1131279766ec49aba2aae37aa315c5a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1131279766ec49aba2aae37aa315c5a7, State: Initialized, Role: FOLLOWER
I20260807 18:14:05.422197 20229 consensus_queue.cc:260] T 42afdb33548544acb0a18f8373a6634b P 1131279766ec49aba2aae37aa315c5a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } }
I20260807 18:14:05.422840 20229 ts_tablet_manager.cc:1428] T 42afdb33548544acb0a18f8373a6634b P 1131279766ec49aba2aae37aa315c5a7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260807 18:14:05.422763 20252 tablet_replica.cc:439] T 42afdb33548544acb0a18f8373a6634b P 1131279766ec49aba2aae37aa315c5a7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } } }
I20260807 18:14:05.423101 20252 tablet_replica.cc:442] T 42afdb33548544acb0a18f8373a6634b P 1131279766ec49aba2aae37aa315c5a7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:14:05.786147 20233 raft_consensus.cc:482] T 42afdb33548544acb0a18f8373a6634b P 8ff251b34c0b429ba9ed605067ceec11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:14:05.786401 20233 raft_consensus.cc:504] T 42afdb33548544acb0a18f8373a6634b P 8ff251b34c0b429ba9ed605067ceec11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } }
I20260807 18:14:05.788471 20233 leader_election.cc:288] T 42afdb33548544acb0a18f8373a6634b P 8ff251b34c0b429ba9ed605067ceec11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1131279766ec49aba2aae37aa315c5a7 (127.17.110.1:35811), aafbbd4e646344f5a4e257344c5f9ddd (127.17.110.2:38765)
I20260807 18:14:05.788545 20252 consensus_queue.cc:1035] T b726a125754b4d11917ddc838d3b7660 P 1131279766ec49aba2aae37aa315c5a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:14:05.804294 20281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:05.788631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:35811 (local address 127.17.110.3:41353)
0807 18:14:05.789356 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:05.789378 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:05.789530 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:05.791254 (+ 1724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:05.791258 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:05.791288 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:14:05.791634 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:05.791643 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:05.793237 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:05.793243 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:05.798548 (+ 5305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:05.798580 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:05.803078 (+ 4498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:05.803089 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:05.803365 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:05.803425 (+ 60us) client_negotiation.cc:769] Sending connection context
0807 18:14:05.803564 (+ 139us) client_negotiation.cc:240] Negotiation successful
0807 18:14:05.803859 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"mutex_wait_us":31,"spinlock_wait_cycles":896,"thread_start_us":666,"threads_started":1}
I20260807 18:14:05.804776 20252 consensus_queue.cc:1035] T b726a125754b4d11917ddc838d3b7660 P 1131279766ec49aba2aae37aa315c5a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:14:05.807900 20283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:05.789176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:41353 (local address 127.17.110.1:35811)
0807 18:14:05.790918 (+ 1742us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:05.790925 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:05.790946 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:14:05.791023 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:05.791028 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:05.791095 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:05.791809 (+ 714us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:14:05.791986 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:05.793043 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:05.800563 (+ 7520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:05.802544 (+ 1981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:05.802809 (+ 265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:05.802928 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:05.804548 (+ 1620us) server_negotiation.cc:299] Negotiation successful
0807 18:14:05.804747 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1629,"thread_start_us":1476,"threads_started":1}
I20260807 18:14:05.808797 19692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42afdb33548544acb0a18f8373a6634b" candidate_uuid: "8ff251b34c0b429ba9ed605067ceec11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1131279766ec49aba2aae37aa315c5a7" is_pre_election: true
I20260807 18:14:05.809262 19692 raft_consensus.cc:2454] T 42afdb33548544acb0a18f8373a6634b P 1131279766ec49aba2aae37aa315c5a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ff251b34c0b429ba9ed605067ceec11 in term 0.
I20260807 18:14:05.809454 20285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:05.789420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:60513 (local address 127.17.110.2:38765)
0807 18:14:05.790677 (+ 1257us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:05.790683 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:05.790732 (+ 49us) server_negotiation.cc:407] Connection header received
0807 18:14:05.790813 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:05.790819 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:05.790895 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:05.790999 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:14:05.791823 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:05.793046 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:05.801758 (+ 8712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:05.803144 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:05.803290 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:05.803359 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:05.808922 (+ 5563us) server_negotiation.cc:299] Negotiation successful
0807 18:14:05.809124 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1113,"thread_start_us":997,"threads_started":1}
I20260807 18:14:05.809927 20019 leader_election.cc:302] T 42afdb33548544acb0a18f8373a6634b P 8ff251b34c0b429ba9ed605067ceec11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1131279766ec49aba2aae37aa315c5a7, 8ff251b34c0b429ba9ed605067ceec11; no voters:
I20260807 18:14:05.810386 20233 raft_consensus.cc:2781] T 42afdb33548544acb0a18f8373a6634b P 8ff251b34c0b429ba9ed605067ceec11 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:14:05.810583 20233 raft_consensus.cc:482] T 42afdb33548544acb0a18f8373a6634b P 8ff251b34c0b429ba9ed605067ceec11 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:14:05.810734 20233 raft_consensus.cc:3037] T 42afdb33548544acb0a18f8373a6634b P 8ff251b34c0b429ba9ed605067ceec11 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:05.815228 20282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:05.788963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:38765 (local address 127.17.110.3:60513)
0807 18:14:05.790030 (+ 1067us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:05.790053 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:05.790198 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:05.791198 (+ 1000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:05.791206 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:05.791238 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:14:05.791618 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:05.791632 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:05.793237 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:05.793243 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:05.801512 (+ 8269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:05.801533 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:05.803590 (+ 2057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:05.803597 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:05.808642 (+ 5045us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:05.808697 (+ 55us) client_negotiation.cc:769] Sending connection context
0807 18:14:05.809661 (+ 964us) client_negotiation.cc:240] Negotiation successful
0807 18:14:05.809838 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":941,"mutex_wait_us":4828,"spinlock_wait_cycles":13056,"thread_start_us":777,"threads_started":1}
I20260807 18:14:05.815990 19902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42afdb33548544acb0a18f8373a6634b" candidate_uuid: "8ff251b34c0b429ba9ed605067ceec11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" is_pre_election: true
I20260807 18:14:05.816303 19902 raft_consensus.cc:2454] T 42afdb33548544acb0a18f8373a6634b P aafbbd4e646344f5a4e257344c5f9ddd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ff251b34c0b429ba9ed605067ceec11 in term 0.
I20260807 18:14:05.816372 20233 raft_consensus.cc:504] T 42afdb33548544acb0a18f8373a6634b P 8ff251b34c0b429ba9ed605067ceec11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } }
I20260807 18:14:05.818157 19692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42afdb33548544acb0a18f8373a6634b" candidate_uuid: "8ff251b34c0b429ba9ed605067ceec11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1131279766ec49aba2aae37aa315c5a7"
I20260807 18:14:05.818477 19692 raft_consensus.cc:3037] T 42afdb33548544acb0a18f8373a6634b P 1131279766ec49aba2aae37aa315c5a7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:05.823145 19692 raft_consensus.cc:2454] T 42afdb33548544acb0a18f8373a6634b P 1131279766ec49aba2aae37aa315c5a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ff251b34c0b429ba9ed605067ceec11 in term 1.
I20260807 18:14:05.823596 20233 leader_election.cc:288] T 42afdb33548544acb0a18f8373a6634b P 8ff251b34c0b429ba9ed605067ceec11 [CANDIDATE]: Term 1 election: Requested vote from peers 1131279766ec49aba2aae37aa315c5a7 (127.17.110.1:35811), aafbbd4e646344f5a4e257344c5f9ddd (127.17.110.2:38765)
I20260807 18:14:05.823720 20019 leader_election.cc:302] T 42afdb33548544acb0a18f8373a6634b P 8ff251b34c0b429ba9ed605067ceec11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1131279766ec49aba2aae37aa315c5a7, 8ff251b34c0b429ba9ed605067ceec11; no voters:
I20260807 18:14:05.823994 19902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42afdb33548544acb0a18f8373a6634b" candidate_uuid: "8ff251b34c0b429ba9ed605067ceec11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aafbbd4e646344f5a4e257344c5f9ddd"
I20260807 18:14:05.824209 20233 raft_consensus.cc:2781] T 42afdb33548544acb0a18f8373a6634b P 8ff251b34c0b429ba9ed605067ceec11 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:14:05.824285 19902 raft_consensus.cc:3037] T 42afdb33548544acb0a18f8373a6634b P aafbbd4e646344f5a4e257344c5f9ddd [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:05.824393 20233 raft_consensus.cc:686] T 42afdb33548544acb0a18f8373a6634b P 8ff251b34c0b429ba9ed605067ceec11 [term 1 LEADER]: Becoming Leader. State: Replica: 8ff251b34c0b429ba9ed605067ceec11, State: Running, Role: LEADER
I20260807 18:14:05.824812 20233 consensus_queue.cc:237] T 42afdb33548544acb0a18f8373a6634b P 8ff251b34c0b429ba9ed605067ceec11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } }
I20260807 18:14:05.827550 19902 raft_consensus.cc:2454] T 42afdb33548544acb0a18f8373a6634b P aafbbd4e646344f5a4e257344c5f9ddd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ff251b34c0b429ba9ed605067ceec11 in term 1.
I20260807 18:14:05.829155 20296 tablet_replica.cc:439] T 42afdb33548544acb0a18f8373a6634b P 8ff251b34c0b429ba9ed605067ceec11: TxnStatusTablet state changed. Reason: New leader 8ff251b34c0b429ba9ed605067ceec11. Latest consensus state: current_term: 1 leader_uuid: "8ff251b34c0b429ba9ed605067ceec11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } } }
I20260807 18:14:05.829686 20296 tablet_replica.cc:442] T 42afdb33548544acb0a18f8373a6634b P 8ff251b34c0b429ba9ed605067ceec11: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:14:05.830911 19458 catalog_manager.cc:5495] T 42afdb33548544acb0a18f8373a6634b P 8ff251b34c0b429ba9ed605067ceec11 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ff251b34c0b429ba9ed605067ceec11 (127.17.110.3). New cstate: current_term: 1 leader_uuid: "8ff251b34c0b429ba9ed605067ceec11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } health_report { overall_health: HEALTHY } } }
I20260807 18:14:05.831276 20298 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 18:14:05.831691 20298 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 18:14:05.886466 20303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:05.878300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:37217 (local address 127.0.0.1:37952)
0807 18:14:05.879469 (+ 1169us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:05.879487 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:05.879654 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:05.880228 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:05.880231 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:05.880241 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:14:05.880441 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:05.880449 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:05.882103 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:05.882116 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:05.883268 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:05.883283 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:05.884386 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:05.884391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:05.884529 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:05.884605 (+ 76us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:14:05.885891 (+ 1286us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:14:05.885919 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 18:14:05.886026 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 18:14:05.886156 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":982,"thread_start_us":847,"threads_started":1}
I20260807 18:14:05.886610 20304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:05.878375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37952 (local address 127.17.110.3:37217)
0807 18:14:05.879162 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:05.879169 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:05.879592 (+ 423us) server_negotiation.cc:407] Connection header received
0807 18:14:05.879872 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:05.879879 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:05.879999 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:05.880201 (+ 202us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:14:05.880614 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:05.881947 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:05.883421 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:05.884184 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:05.884335 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:05.884855 (+ 520us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:14:05.885716 (+ 861us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:14:05.885878 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:05.886091 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 18:14:05.886191 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":450,"threads_started":1}
I20260807 18:14:05.892391 19902 raft_consensus.cc:1261] T 42afdb33548544acb0a18f8373a6634b P aafbbd4e646344f5a4e257344c5f9ddd [term 1 FOLLOWER]: Refusing update from remote peer 8ff251b34c0b429ba9ed605067ceec11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:14:05.892678 19692 raft_consensus.cc:1261] T 42afdb33548544acb0a18f8373a6634b P 1131279766ec49aba2aae37aa315c5a7 [term 1 FOLLOWER]: Refusing update from remote peer 8ff251b34c0b429ba9ed605067ceec11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:14:05.893450 20296 consensus_queue.cc:1035] T 42afdb33548544acb0a18f8373a6634b P 8ff251b34c0b429ba9ed605067ceec11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:14:05.893827 20233 consensus_queue.cc:1035] T 42afdb33548544acb0a18f8373a6634b P 8ff251b34c0b429ba9ed605067ceec11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:14:05.899813 20235 tablet_replica.cc:439] T 42afdb33548544acb0a18f8373a6634b P aafbbd4e646344f5a4e257344c5f9ddd: TxnStatusTablet state changed. Reason: New leader 8ff251b34c0b429ba9ed605067ceec11. Latest consensus state: current_term: 1 leader_uuid: "8ff251b34c0b429ba9ed605067ceec11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } } }
I20260807 18:14:05.900314 20235 tablet_replica.cc:442] T 42afdb33548544acb0a18f8373a6634b P aafbbd4e646344f5a4e257344c5f9ddd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:14:05.902462 20293 tablet_replica.cc:439] T 42afdb33548544acb0a18f8373a6634b P 1131279766ec49aba2aae37aa315c5a7: TxnStatusTablet state changed. Reason: New leader 8ff251b34c0b429ba9ed605067ceec11. Latest consensus state: current_term: 1 leader_uuid: "8ff251b34c0b429ba9ed605067ceec11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } } }
I20260807 18:14:05.903800 20293 tablet_replica.cc:442] T 42afdb33548544acb0a18f8373a6634b P 1131279766ec49aba2aae37aa315c5a7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:14:05.906443 20290 mvcc.cc:205] Tried to move back new op lower bound from 7315973922365947904 to 7315973922103123968. Current Snapshot: MvccSnapshot[applied={T|T < 7315973922365947904}]
I20260807 18:14:05.907734 20296 tablet_replica.cc:439] T 42afdb33548544acb0a18f8373a6634b P 8ff251b34c0b429ba9ed605067ceec11: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8ff251b34c0b429ba9ed605067ceec11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } } }
I20260807 18:14:05.908053 20296 tablet_replica.cc:442] T 42afdb33548544acb0a18f8373a6634b P 8ff251b34c0b429ba9ed605067ceec11: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:14:05.909417 20296 tablet_replica.cc:439] T 42afdb33548544acb0a18f8373a6634b P 8ff251b34c0b429ba9ed605067ceec11: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8ff251b34c0b429ba9ed605067ceec11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } } }
I20260807 18:14:05.909710 20296 tablet_replica.cc:442] T 42afdb33548544acb0a18f8373a6634b P 8ff251b34c0b429ba9ed605067ceec11: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:14:05.912699 20293 tablet_replica.cc:439] T 42afdb33548544acb0a18f8373a6634b P 1131279766ec49aba2aae37aa315c5a7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8ff251b34c0b429ba9ed605067ceec11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } } }
I20260807 18:14:05.912987 20293 tablet_replica.cc:442] T 42afdb33548544acb0a18f8373a6634b P 1131279766ec49aba2aae37aa315c5a7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:14:05.916864 20235 tablet_replica.cc:439] T 42afdb33548544acb0a18f8373a6634b P aafbbd4e646344f5a4e257344c5f9ddd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8ff251b34c0b429ba9ed605067ceec11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1131279766ec49aba2aae37aa315c5a7" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35811 } } peers { permanent_uuid: "aafbbd4e646344f5a4e257344c5f9ddd" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38765 } } peers { permanent_uuid: "8ff251b34c0b429ba9ed605067ceec11" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37217 } } }
I20260807 18:14:05.917151 20235 tablet_replica.cc:442] T 42afdb33548544acb0a18f8373a6634b P aafbbd4e646344f5a4e257344c5f9ddd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:14:05.940632 20304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:05.930397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:47819 (local address 127.17.110.3:37217)
0807 18:14:05.930642 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:05.930649 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:05.932113 (+ 1464us) server_negotiation.cc:407] Connection header received
0807 18:14:05.932490 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:05.932497 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:05.932618 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:05.932724 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:14:05.933236 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:05.934384 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:05.936665 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:05.937419 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:05.937758 (+ 339us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:05.939589 (+ 1831us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:14:05.939800 (+ 211us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:14:05.939928 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:05.940130 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 18:14:05.940232 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 18:14:05.940732 20315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:05.930342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:37217 (local address 127.17.110.1:47819)
0807 18:14:05.931809 (+ 1467us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:05.931828 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:05.932253 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:05.932793 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:05.932797 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:05.932808 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:14:05.933071 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:05.933080 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:05.934896 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:05.934901 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:05.936409 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:05.936430 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:05.938213 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:05.938227 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:05.938539 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:05.938638 (+ 99us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:14:05.939954 (+ 1316us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:14:05.939977 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 18:14:05.940078 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 18:14:05.940278 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1233,"thread_start_us":1053,"threads_started":1}
I20260807 18:14:06.002262 20283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:05.993278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:47609 (local address 127.17.110.1:35811)
0807 18:14:05.993795 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:05.993802 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:05.994337 (+ 535us) server_negotiation.cc:407] Connection header received
0807 18:14:05.994574 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:05.994580 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:05.994684 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:05.994826 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:14:05.995638 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:05.996840 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:05.999608 (+ 2768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:06.000320 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:06.000486 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:06.001411 (+ 925us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:14:06.001547 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:14:06.001663 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:06.001838 (+ 175us) server_negotiation.cc:299] Negotiation successful
0807 18:14:06.001932 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298}
I20260807 18:14:06.002450 20322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:05.993245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:35811 (local address 127.17.110.3:47609)
0807 18:14:05.994162 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:05.994187 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:05.994380 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:05.994854 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:05.994861 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:05.994876 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:14:05.995398 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:05.995415 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:05.997026 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:05.997034 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:05.999365 (+ 2331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:05.999393 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:06.000520 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:06.000531 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:06.000904 (+ 373us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:06.001010 (+ 106us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:14:06.001667 (+ 657us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:14:06.001685 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 18:14:06.001801 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 18:14:06.001976 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":478,"threads_started":1}
I20260807 18:14:06.067800 19458 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:14:06.068185 19458 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:14:06.072899 19458 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ca311ba787804c1b9e854c216d9e7289: Sending DeleteTablet for 3 replicas of tablet b726a125754b4d11917ddc838d3b7660
I20260807 18:14:06.074070 19656 tablet_service.cc:1514] Processing DeleteTablet for tablet b726a125754b4d11917ddc838d3b7660 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:14:06 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:57802
I20260807 18:14:06.074048 19875 tablet_service.cc:1514] Processing DeleteTablet for tablet b726a125754b4d11917ddc838d3b7660 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:14:06 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:50322
I20260807 18:14:06.074949 20088 tablet_service.cc:1514] Processing DeleteTablet for tablet b726a125754b4d11917ddc838d3b7660 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:14:06 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:37946
I20260807 18:14:06.085791 20329 tablet_replica.cc:330] T b726a125754b4d11917ddc838d3b7660 P 8ff251b34c0b429ba9ed605067ceec11: stopping tablet replica
I20260807 18:14:06.086361 20329 raft_consensus.cc:2229] T b726a125754b4d11917ddc838d3b7660 P 8ff251b34c0b429ba9ed605067ceec11 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:14:06.087141 20325 tablet_replica.cc:330] T b726a125754b4d11917ddc838d3b7660 P aafbbd4e646344f5a4e257344c5f9ddd: stopping tablet replica
I20260807 18:14:06.087430 20328 tablet_replica.cc:330] T b726a125754b4d11917ddc838d3b7660 P 1131279766ec49aba2aae37aa315c5a7: stopping tablet replica
I20260807 18:14:06.087598 20329 raft_consensus.cc:2258] T b726a125754b4d11917ddc838d3b7660 P 8ff251b34c0b429ba9ed605067ceec11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:14:06.087730 20328 raft_consensus.cc:2229] T b726a125754b4d11917ddc838d3b7660 P 1131279766ec49aba2aae37aa315c5a7 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:14:06.088083 20325 raft_consensus.cc:2229] T b726a125754b4d11917ddc838d3b7660 P aafbbd4e646344f5a4e257344c5f9ddd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:14:06.088244 20328 raft_consensus.cc:2258] T b726a125754b4d11917ddc838d3b7660 P 1131279766ec49aba2aae37aa315c5a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:14:06.088429 20325 raft_consensus.cc:2258] T b726a125754b4d11917ddc838d3b7660 P aafbbd4e646344f5a4e257344c5f9ddd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:14:06.089502 20328 ts_tablet_manager.cc:1905] T b726a125754b4d11917ddc838d3b7660 P 1131279766ec49aba2aae37aa315c5a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:14:06.090126 20329 ts_tablet_manager.cc:1905] T b726a125754b4d11917ddc838d3b7660 P 8ff251b34c0b429ba9ed605067ceec11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:14:06.090302 20325 ts_tablet_manager.cc:1905] T b726a125754b4d11917ddc838d3b7660 P aafbbd4e646344f5a4e257344c5f9ddd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:14:06.099067 20329 ts_tablet_manager.cc:1918] T b726a125754b4d11917ddc838d3b7660 P 8ff251b34c0b429ba9ed605067ceec11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:14:06.099339 20329 log.cc:1196] T b726a125754b4d11917ddc838d3b7660 P 8ff251b34c0b429ba9ed605067ceec11: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-2/wal/wals/b726a125754b4d11917ddc838d3b7660
I20260807 18:14:06.100162 20329 ts_tablet_manager.cc:1939] T b726a125754b4d11917ddc838d3b7660 P 8ff251b34c0b429ba9ed605067ceec11: Deleting consensus metadata
I20260807 18:14:06.100687 20328 ts_tablet_manager.cc:1918] T b726a125754b4d11917ddc838d3b7660 P 1131279766ec49aba2aae37aa315c5a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:14:06.100859 20328 log.cc:1196] T b726a125754b4d11917ddc838d3b7660 P 1131279766ec49aba2aae37aa315c5a7: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-0/wal/wals/b726a125754b4d11917ddc838d3b7660
I20260807 18:14:06.101351 20328 ts_tablet_manager.cc:1939] T b726a125754b4d11917ddc838d3b7660 P 1131279766ec49aba2aae37aa315c5a7: Deleting consensus metadata
I20260807 18:14:06.101933 20325 ts_tablet_manager.cc:1918] T b726a125754b4d11917ddc838d3b7660 P aafbbd4e646344f5a4e257344c5f9ddd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:14:06.101982 19440 catalog_manager.cc:4841] TS 8ff251b34c0b429ba9ed605067ceec11 (127.17.110.3:37217): tablet b726a125754b4d11917ddc838d3b7660 (table test-table [id=28ad03bcea8d4b419cdddcb506f105d7]) successfully deleted
I20260807 18:14:06.102101 20325 log.cc:1196] T b726a125754b4d11917ddc838d3b7660 P aafbbd4e646344f5a4e257344c5f9ddd: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126429568977-17848-0/minicluster-data/ts-1/wal/wals/b726a125754b4d11917ddc838d3b7660
I20260807 18:14:06.102661 20325 ts_tablet_manager.cc:1939] T b726a125754b4d11917ddc838d3b7660 P aafbbd4e646344f5a4e257344c5f9ddd: Deleting consensus metadata
I20260807 18:14:06.102661 19443 catalog_manager.cc:4841] TS 1131279766ec49aba2aae37aa315c5a7 (127.17.110.1:35811): tablet b726a125754b4d11917ddc838d3b7660 (table test-table [id=28ad03bcea8d4b419cdddcb506f105d7]) successfully deleted
I20260807 18:14:06.103842 19440 catalog_manager.cc:4841] TS aafbbd4e646344f5a4e257344c5f9ddd (127.17.110.2:38765): tablet b726a125754b4d11917ddc838d3b7660 (table test-table [id=28ad03bcea8d4b419cdddcb506f105d7]) successfully deleted
Aug 07 18:14:06 dist-test-slave-39l8 krb5kdc[19400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126444, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.110.1@KRBTEST.COM
I20260807 18:14:06.119158 20283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:06.089371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57820 (local address 127.17.110.1:35811)
0807 18:14:06.089597 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:06.089603 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:06.097946 (+ 8343us) server_negotiation.cc:407] Connection header received
0807 18:14:06.098271 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:06.098281 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:06.098377 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:06.098452 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:06.103767 (+ 5315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:06.104748 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:06.106209 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:06.106653 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:06.106894 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:06.112135 (+ 5241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:06.112153 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:06.112171 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:06.112225 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:06.115739 (+ 3514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:06.116662 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:06.116671 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:06.116673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:06.116841 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:06.117320 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:06.117324 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:06.117326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:06.117753 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:06.118101 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:06.118456 (+ 355us) server_negotiation.cc:299] Negotiation successful
0807 18:14:06.118776 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260807 18:14:06.119976 19756 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57820
I20260807 18:14:06.151826 20336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:06.123829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45548 (local address 127.17.110.62:34857)
0807 18:14:06.132148 (+ 8319us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:06.132154 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:06.132207 (+ 53us) server_negotiation.cc:407] Connection header received
0807 18:14:06.132279 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:06.132283 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:06.132362 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:06.132504 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:06.134943 (+ 2439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:06.135875 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:06.138028 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:06.138599 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:06.138845 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:06.142657 (+ 3812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:06.142664 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:06.142667 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:06.142807 (+ 140us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:06.147311 (+ 4504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:06.148407 (+ 1096us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:06.148414 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:06.148420 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:06.148686 (+ 266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:06.149262 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:06.149268 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:06.149271 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:06.149699 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:06.150015 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:06.151125 (+ 1110us) server_negotiation.cc:299] Negotiation successful
0807 18:14:06.151423 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8056,"thread_start_us":7713,"threads_started":1}
W20260807 18:14:06.152666 19458 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45548
I20260807 18:14:06.155397 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 19561
I20260807 18:14:06.174696 20283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:06.156505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57830 (local address 127.17.110.1:35811)
0807 18:14:06.156744 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:06.156754 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:06.157091 (+ 337us) server_negotiation.cc:407] Connection header received
0807 18:14:06.157329 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:06.157343 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:06.157416 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:06.157539 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:06.159071 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:06.159898 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:06.161341 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:06.162375 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:06.162562 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:06.166557 (+ 3995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:06.166564 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:06.166568 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:06.166619 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:06.170934 (+ 4315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:06.172375 (+ 1441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:06.172383 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:06.172387 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:06.172588 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:06.173075 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:06.173080 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:06.173082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:06.173403 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:06.173688 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:06.174085 (+ 397us) server_negotiation.cc:299] Negotiation successful
0807 18:14:06.174353 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260807 18:14:06.175405 19756 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57830
W20260807 18:14:06.176152 17848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 19561: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:14:06.176409 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 19561
I20260807 18:14:06.229391 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 19775
Aug 07 18:14:06 dist-test-slave-39l8 krb5kdc[19400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126444, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.110.2@KRBTEST.COM
I20260807 18:14:06.253898 20285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:06.230452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50332 (local address 127.17.110.2:38765)
0807 18:14:06.230728 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:06.230736 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:06.231105 (+ 369us) server_negotiation.cc:407] Connection header received
0807 18:14:06.231647 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:06.231653 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:06.231755 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:06.231882 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:06.233646 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:06.235033 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:06.237246 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:06.237935 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:06.238144 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:06.244459 (+ 6315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:06.244486 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:06.244511 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:06.244574 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:06.249621 (+ 5047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:06.250784 (+ 1163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:06.250796 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:06.250802 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:06.251010 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:06.251572 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:06.251579 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:06.251582 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:06.252434 (+ 852us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:06.252814 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:06.253217 (+ 403us) server_negotiation.cc:299] Negotiation successful
0807 18:14:06.253517 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260807 18:14:06.254952 19976 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50332
W20260807 18:14:06.255880 17848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 19775: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:14:06.256108 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 19775
I20260807 18:14:06.286852 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 19988
Aug 07 18:14:06 dist-test-slave-39l8 krb5kdc[19400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126444, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.110.3@KRBTEST.COM
I20260807 18:14:06.308503 20304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:06.287806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37964 (local address 127.17.110.3:37217)
0807 18:14:06.288028 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:06.288033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:06.288055 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:14:06.288408 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:06.288413 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:06.288515 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:06.288614 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:06.290568 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:06.291708 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:06.293861 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:06.294416 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:06.294531 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:06.301067 (+ 6536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:06.301085 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:06.301103 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:06.301155 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:06.304706 (+ 3551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:06.305652 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:06.305661 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:06.305664 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:06.305836 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:06.306290 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:06.306305 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:06.306307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:06.306860 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:06.307190 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:06.307793 (+ 603us) server_negotiation.cc:299] Negotiation successful
0807 18:14:06.308035 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260807 18:14:06.309368 20195 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37964
W20260807 18:14:06.310217 17848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 19988: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:14:06.310371 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 19988
I20260807 18:14:06.342991 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 19418
I20260807 18:14:06.344456 19517 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:14:06.468896 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 19418
2026-08-07T18:14:06Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (6791 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:14:06 dist-test-slave-39l8 krb5kdc[20345](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:14:06 dist-test-slave-39l8 krb5kdc[20345](info): set up 2 sockets
Aug 07 18:14:06 dist-test-slave-39l8 krb5kdc[20345](info): commencing operation
krb5kdc: starting...
W20260807 18:14:07.841188 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.311s user 0.004s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:14:08 dist-test-slave-39l8 krb5kdc[20345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:14:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:14:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.110.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
I20260807 18:14:08.100066 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:44799
--webserver_interface=127.17.110.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:39545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:44799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:14:08.507583 20363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:08.508034 20363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:08.508215 20363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:08.516896 20363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:14:08.517122 20363 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 18:14:08.517239 20363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:08.517354 20363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:08.517441 20363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:14:08.517546 20363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:14:08.529094 20363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:39545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:44799
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:44799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:08.530609 20363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:08.532111 20363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:08.545830 20374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:08.545877 20372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:08.545840 20371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:08.547148 20363 server_base.cc:1034] running on GCE node
I20260807 18:14:08.548089 20363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:08.549888 20363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:08.551232 20363 hybrid_clock.cc:648] HybridClock initialized: now 1786126448551160 us; error 58 us; skew 500 ppm
I20260807 18:14:08.551657 20363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:08 dist-test-slave-39l8 krb5kdc[20345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126448, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:08.555711 20363 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:14:08.557619 20363 webserver.cc:460] Webserver started at http://127.17.110.62:43327/ using document root <none> and password file <none>
I20260807 18:14:08.558917 20363 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:08.559129 20363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:08.559535 20363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:08.563495 20363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "d176896ac4b84f47add8a59c25daceec"
format_stamp: "Formatted at 2026-08-07 18:14:08 on dist-test-slave-39l8"
server_key: "9c8ccaa52bf97a369f8bd0ed8b753e20"
server_key_iv: "6fd385cd13defdcf7cea9206b793b05d"
server_key_version: "encryptionkey@0"
I20260807 18:14:08.564718 20363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "d176896ac4b84f47add8a59c25daceec"
format_stamp: "Formatted at 2026-08-07 18:14:08 on dist-test-slave-39l8"
server_key: "9c8ccaa52bf97a369f8bd0ed8b753e20"
server_key_iv: "6fd385cd13defdcf7cea9206b793b05d"
server_key_version: "encryptionkey@0"
I20260807 18:14:08.571955 20363 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260807 18:14:08.576745 20384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:08.578668 20363 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:14:08.579033 20363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "d176896ac4b84f47add8a59c25daceec"
format_stamp: "Formatted at 2026-08-07 18:14:08 on dist-test-slave-39l8"
server_key: "9c8ccaa52bf97a369f8bd0ed8b753e20"
server_key_iv: "6fd385cd13defdcf7cea9206b793b05d"
server_key_version: "encryptionkey@0"
I20260807 18:14:08.579358 20363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:08.599411 20363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:08.604944 20363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:08.605285 20363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:08.633626 20363 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:44799
I20260807 18:14:08.633648 20472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:44799 every 8 connection(s)
I20260807 18:14:08.635324 20363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:14:08.639986 20473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:08.641856 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 20363
I20260807 18:14:08.642112 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:14:08.643095 17848 external_mini_cluster.cc:1428] Setting key b6a6e08f01d3501cb5a1fac7a15f140a
I20260807 18:14:08.655177 20473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d176896ac4b84f47add8a59c25daceec: Bootstrap starting.
I20260807 18:14:08.659982 20473 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d176896ac4b84f47add8a59c25daceec: Neither blocks nor log segments found. Creating new log.
Aug 07 18:14:08 dist-test-slave-39l8 krb5kdc[20345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:08.661661 20473 log.cc:824] T 00000000000000000000000000000000 P d176896ac4b84f47add8a59c25daceec: Log is configured to *not* fsync() on all Append() calls
I20260807 18:14:08.666431 20473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d176896ac4b84f47add8a59c25daceec: No bootstrap required, opened a new log
I20260807 18:14:08.674585 20476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:08.645634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42426 (local address 127.17.110.62:44799)
0807 18:14:08.646752 (+ 1118us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:08.646775 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:08.646836 (+ 61us) server_negotiation.cc:407] Connection header received
0807 18:14:08.648792 (+ 1956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:08.648821 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:08.649302 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:08.649707 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:08.652024 (+ 2317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:08.653783 (+ 1759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:08.655825 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:08.656691 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:08.656875 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:08.663069 (+ 6194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:08.663080 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:08.663151 (+ 71us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:08.663208 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:08.668381 (+ 5173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:08.669438 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:08.669453 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:08.669464 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:08.669771 (+ 307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:08.670328 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:08.670339 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:08.670344 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:08.671380 (+ 1036us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:08.671773 (+ 393us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:08.672254 (+ 481us) server_negotiation.cc:299] Negotiation successful
0807 18:14:08.672821 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":783,"thread_start_us":501,"threads_started":1}
I20260807 18:14:08.678262 20473 raft_consensus.cc:348] T 00000000000000000000000000000000 P d176896ac4b84f47add8a59c25daceec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d176896ac4b84f47add8a59c25daceec" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 44799 } }
I20260807 18:14:08.678776 20473 raft_consensus.cc:374] T 00000000000000000000000000000000 P d176896ac4b84f47add8a59c25daceec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:08.678893 20473 raft_consensus.cc:729] T 00000000000000000000000000000000 P d176896ac4b84f47add8a59c25daceec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d176896ac4b84f47add8a59c25daceec, State: Initialized, Role: FOLLOWER
I20260807 18:14:08.679581 20473 consensus_queue.cc:260] T 00000000000000000000000000000000 P d176896ac4b84f47add8a59c25daceec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d176896ac4b84f47add8a59c25daceec" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 44799 } }
I20260807 18:14:08.679855 20473 raft_consensus.cc:388] T 00000000000000000000000000000000 P d176896ac4b84f47add8a59c25daceec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:14:08.679975 20473 raft_consensus.cc:482] T 00000000000000000000000000000000 P d176896ac4b84f47add8a59c25daceec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:14:08.680126 20473 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d176896ac4b84f47add8a59c25daceec [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:08.682971 20473 raft_consensus.cc:504] T 00000000000000000000000000000000 P d176896ac4b84f47add8a59c25daceec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d176896ac4b84f47add8a59c25daceec" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 44799 } }
I20260807 18:14:08.683835 20473 leader_election.cc:302] T 00000000000000000000000000000000 P d176896ac4b84f47add8a59c25daceec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d176896ac4b84f47add8a59c25daceec; no voters:
I20260807 18:14:08.684655 20473 leader_election.cc:288] T 00000000000000000000000000000000 P d176896ac4b84f47add8a59c25daceec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:14:08.684829 20481 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d176896ac4b84f47add8a59c25daceec [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:14:08.685735 20481 raft_consensus.cc:686] T 00000000000000000000000000000000 P d176896ac4b84f47add8a59c25daceec [term 1 LEADER]: Becoming Leader. State: Replica: d176896ac4b84f47add8a59c25daceec, State: Running, Role: LEADER
I20260807 18:14:08.686396 20481 consensus_queue.cc:237] T 00000000000000000000000000000000 P d176896ac4b84f47add8a59c25daceec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d176896ac4b84f47add8a59c25daceec" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 44799 } }
I20260807 18:14:08.687417 20473 sys_catalog.cc:564] T 00000000000000000000000000000000 P d176896ac4b84f47add8a59c25daceec [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:14:08.689556 20483 sys_catalog.cc:455] T 00000000000000000000000000000000 P d176896ac4b84f47add8a59c25daceec [sys.catalog]: SysCatalogTable state changed. Reason: New leader d176896ac4b84f47add8a59c25daceec. Latest consensus state: current_term: 1 leader_uuid: "d176896ac4b84f47add8a59c25daceec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d176896ac4b84f47add8a59c25daceec" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 44799 } } }
I20260807 18:14:08.689975 20483 sys_catalog.cc:458] T 00000000000000000000000000000000 P d176896ac4b84f47add8a59c25daceec [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:08.691257 20491 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:14:08.693017 20482 sys_catalog.cc:455] T 00000000000000000000000000000000 P d176896ac4b84f47add8a59c25daceec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d176896ac4b84f47add8a59c25daceec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d176896ac4b84f47add8a59c25daceec" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 44799 } } }
I20260807 18:14:08.693426 20482 sys_catalog.cc:458] T 00000000000000000000000000000000 P d176896ac4b84f47add8a59c25daceec [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:08.701279 20491 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:14:08.714251 20491 catalog_manager.cc:1347] Generated new cluster ID: 6fc39cd8a5f24e7b8f393fa2752bf28c
I20260807 18:14:08.714372 20491 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:14:08.748799 20491 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:14:08.751168 20491 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:14:08.762625 20491 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d176896ac4b84f47add8a59c25daceec: Generated new TSK 0
I20260807 18:14:08.763682 20491 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.110.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
I20260807 18:14:08.832855 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.1:0
--local_ip_for_outbound_sockets=127.17.110.1
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:44799
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:39545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:14:09.222858 20511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:09.223232 20511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:09.223522 20511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:09.232259 20511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:09.232410 20511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:09.232569 20511 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 18:14:09.232648 20511 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:14:09.232769 20511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.1
I20260807 18:14:09.244047 20511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:39545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.110.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.110.62:44799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.1
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:09.245954 20511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:09.247709 20511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:09.264046 20516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:09.264148 20517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:09.264097 20521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:09.266446 20511 server_base.cc:1034] running on GCE node
I20260807 18:14:09.267343 20511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:09.268868 20511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:09.270220 20511 hybrid_clock.cc:648] HybridClock initialized: now 1786126449270147 us; error 55 us; skew 500 ppm
I20260807 18:14:09.270661 20511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:09 dist-test-slave-39l8 krb5kdc[20345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126449, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:09.279546 20511 init.cc:377] Logged in from keytab as kudu/127.17.110.1@KRBTEST.COM (short username kudu)
I20260807 18:14:09.281544 20511 webserver.cc:460] Webserver started at http://127.17.110.1:43045/ using document root <none> and password file <none>
I20260807 18:14:09.282346 20511 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:09.282469 20511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:09.282950 20511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:09.286830 20511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-0/data/instance:
uuid: "067119311361489a9621056d34ddce20"
format_stamp: "Formatted at 2026-08-07 18:14:09 on dist-test-slave-39l8"
server_key: "cde5bbd8aac8bb51d29c9e62a73b85a8"
server_key_iv: "4c94f022657394d825c046b16b006465"
server_key_version: "encryptionkey@0"
I20260807 18:14:09.287842 20511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance:
uuid: "067119311361489a9621056d34ddce20"
format_stamp: "Formatted at 2026-08-07 18:14:09 on dist-test-slave-39l8"
server_key: "cde5bbd8aac8bb51d29c9e62a73b85a8"
server_key_iv: "4c94f022657394d825c046b16b006465"
server_key_version: "encryptionkey@0"
I20260807 18:14:09.295187 20511 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.002s
I20260807 18:14:09.300094 20531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:09.301961 20511 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:14:09.302232 20511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-0/wal
uuid: "067119311361489a9621056d34ddce20"
format_stamp: "Formatted at 2026-08-07 18:14:09 on dist-test-slave-39l8"
server_key: "cde5bbd8aac8bb51d29c9e62a73b85a8"
server_key_iv: "4c94f022657394d825c046b16b006465"
server_key_version: "encryptionkey@0"
I20260807 18:14:09.302605 20511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:09.319290 20511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:09.324541 20511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:09.324895 20511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:09.328118 20511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:14:09.328271 20511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:09.328444 20511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:14:09.328552 20511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:14:09 dist-test-slave-39l8 krb5kdc[20345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126449, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:09.357677 20545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:09.329009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:47529 (local address 127.17.110.62:44799)
0807 18:14:09.329617 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:09.329622 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:09.330838 (+ 1216us) server_negotiation.cc:407] Connection header received
0807 18:14:09.333179 (+ 2341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:09.333184 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:09.333303 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:09.333433 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:09.336198 (+ 2765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:09.337136 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:09.340137 (+ 3001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:09.340720 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:09.340876 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:09.347314 (+ 6438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:09.347325 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:09.347332 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:09.347393 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:09.351899 (+ 4506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:09.353866 (+ 1967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:09.353877 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:09.353884 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:09.354118 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:09.354927 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:09.354935 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:09.354943 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:09.355475 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:09.355663 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:09.356964 (+ 1301us) server_negotiation.cc:299] Negotiation successful
0807 18:14:09.357266 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":366,"threads_started":1}
I20260807 18:14:09.359023 20546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:09.329610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:44799 (local address 127.17.110.1:47529)
0807 18:14:09.330583 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:09.330627 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:09.332831 (+ 2204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:09.333799 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:09.333814 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:09.334830 (+ 1016us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:09.335856 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:09.335898 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:09.337351 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:09.337365 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:09.339908 (+ 2543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:09.339931 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:09.341201 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:09.341217 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:09.341584 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:09.342682 (+ 1098us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:09.342761 (+ 79us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:09.346894 (+ 4133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:09.352245 (+ 5351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:09.352263 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:09.352289 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:09.353580 (+ 1291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:09.354286 (+ 706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:09.354292 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:09.354302 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:09.354769 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:09.355647 (+ 878us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:09.355658 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:09.356360 (+ 702us) client_negotiation.cc:769] Sending connection context
0807 18:14:09.356838 (+ 478us) client_negotiation.cc:240] Negotiation successful
0807 18:14:09.357509 (+ 671us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":413,"threads_started":1}
I20260807 18:14:09.380781 20511 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.1:35891
I20260807 18:14:09.380801 20711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.1:35891 every 8 connection(s)
I20260807 18:14:09.382296 20511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
I20260807 18:14:09.385286 20678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:09.367239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:44799 (local address 127.17.110.1:38937)
0807 18:14:09.367909 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:09.367929 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:09.368084 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:09.368576 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:09.368585 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:09.369522 (+ 937us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:09.370200 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:09.370229 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:09.372235 (+ 2006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:09.372249 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:09.373724 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:09.373745 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:09.374788 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:09.374797 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:09.374987 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:09.375961 (+ 974us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:09.375981 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:09.377269 (+ 1288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:09.381455 (+ 4186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:09.381462 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:09.381466 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:09.381976 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:09.382501 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:09.382505 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:09.382507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:09.382652 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:09.383788 (+ 1136us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:09.383794 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:09.384087 (+ 293us) client_negotiation.cc:769] Sending connection context
0807 18:14:09.384282 (+ 195us) client_negotiation.cc:240] Negotiation successful
0807 18:14:09.384755 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":351,"threads_started":1}
I20260807 18:14:09.385350 20712 heartbeater.cc:347] Connected to a master server at 127.17.110.62:44799
I20260807 18:14:09.385735 20712 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:09.386379 20545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:09.367219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:38937 (local address 127.17.110.62:44799)
0807 18:14:09.367417 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:09.367422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:09.368048 (+ 626us) server_negotiation.cc:407] Connection header received
0807 18:14:09.368209 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:09.368215 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:09.368320 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:09.368454 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:09.370524 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:09.371886 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:09.373962 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:09.374493 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:09.374687 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:09.377743 (+ 3056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:09.377752 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:09.377757 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:09.377804 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:09.381248 (+ 3444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:09.382139 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:09.382152 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:09.382155 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:09.382310 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:09.382811 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:09.382819 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:09.382822 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:09.383593 (+ 771us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:09.385664 (+ 2071us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:09.385842 (+ 178us) server_negotiation.cc:299] Negotiation successful
0807 18:14:09.386096 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:14:09.386600 20712 heartbeater.cc:511] Master 127.17.110.62:44799 requested a full tablet report, sending...
I20260807 18:14:09.389541 20409 ts_manager.cc:194] Registered new tserver with Master: 067119311361489a9621056d34ddce20 (127.17.110.1:35891)
I20260807 18:14:09.389797 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 20511
I20260807 18:14:09.390010 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance
I20260807 18:14:09.390885 17848 external_mini_cluster.cc:1428] Setting key e7cf91f280e2917bf8b6b4488d11af82
I20260807 18:14:09.391556 20409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.1@KRBTEST.COM'} at 127.17.110.1:47529
WARNING: no policy specified for kudu/127.17.110.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
I20260807 18:14:09.442580 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.2:0
--local_ip_for_outbound_sockets=127.17.110.2
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:44799
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:39545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:14:09.824807 20718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:09.825214 20718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:09.825403 20718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:09.833364 20718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:09.833557 20718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:09.833731 20718 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 18:14:09.833823 20718 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:14:09.833969 20718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.2
I20260807 18:14:09.844664 20718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:39545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.110.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.110.62:44799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.2
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:09.846243 20718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:09.847806 20718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:09.863283 20726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:09.863243 20727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:09.863305 20730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:09.864387 20718 server_base.cc:1034] running on GCE node
I20260807 18:14:09.865082 20718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:09.866140 20718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:09.867372 20718 hybrid_clock.cc:648] HybridClock initialized: now 1786126449867323 us; error 45 us; skew 500 ppm
I20260807 18:14:09.867748 20718 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:09 dist-test-slave-39l8 krb5kdc[20345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126449, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:09.871937 20718 init.cc:377] Logged in from keytab as kudu/127.17.110.2@KRBTEST.COM (short username kudu)
I20260807 18:14:09.873879 20718 webserver.cc:460] Webserver started at http://127.17.110.2:37729/ using document root <none> and password file <none>
I20260807 18:14:09.874763 20718 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:09.874945 20718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:09.875311 20718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:09.878594 20718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-1/data/instance:
uuid: "aa235ba635bf4b5ca63e16ce4aabed61"
format_stamp: "Formatted at 2026-08-07 18:14:09 on dist-test-slave-39l8"
server_key: "818f25caf2ac569c92a808041a93b4d7"
server_key_iv: "af24a61bc98e347f2157f91a4deba36e"
server_key_version: "encryptionkey@0"
I20260807 18:14:09.879591 20718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance:
uuid: "aa235ba635bf4b5ca63e16ce4aabed61"
format_stamp: "Formatted at 2026-08-07 18:14:09 on dist-test-slave-39l8"
server_key: "818f25caf2ac569c92a808041a93b4d7"
server_key_iv: "af24a61bc98e347f2157f91a4deba36e"
server_key_version: "encryptionkey@0"
I20260807 18:14:09.886260 20718 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 18:14:09.891362 20740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:09.893270 20718 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:14:09.893471 20718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-1/wal
uuid: "aa235ba635bf4b5ca63e16ce4aabed61"
format_stamp: "Formatted at 2026-08-07 18:14:09 on dist-test-slave-39l8"
server_key: "818f25caf2ac569c92a808041a93b4d7"
server_key_iv: "af24a61bc98e347f2157f91a4deba36e"
server_key_version: "encryptionkey@0"
I20260807 18:14:09.893698 20718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:09.913822 20718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:09.919595 20718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:09.919916 20718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:09.923177 20718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:14:09.923323 20718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:09.923468 20718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:14:09.923560 20718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:14:09 dist-test-slave-39l8 krb5kdc[20345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126449, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:09.952682 20752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:09.924225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:45071 (local address 127.17.110.62:44799)
0807 18:14:09.924813 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:09.924819 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:09.926225 (+ 1406us) server_negotiation.cc:407] Connection header received
0807 18:14:09.928754 (+ 2529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:09.928768 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:09.928883 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:09.929053 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:09.932252 (+ 3199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:09.933819 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:09.936615 (+ 2796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:09.937238 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:09.937440 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:09.944140 (+ 6700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:09.944148 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:09.944152 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:09.944201 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:09.947683 (+ 3482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:09.948954 (+ 1271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:09.948965 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:09.948968 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:09.949203 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:09.950088 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:09.950096 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:09.950101 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:09.950522 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:09.950777 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:09.951933 (+ 1156us) server_negotiation.cc:299] Negotiation successful
0807 18:14:09.952252 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":345,"threads_started":1}
I20260807 18:14:09.954587 20753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:09.925012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:44799 (local address 127.17.110.2:45071)
0807 18:14:09.926003 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:09.926058 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:09.928315 (+ 2257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:09.929561 (+ 1246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:09.929571 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:09.930554 (+ 983us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:09.931770 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:09.931799 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:09.934175 (+ 2376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:09.934180 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:09.936365 (+ 2185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:09.936395 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:09.937594 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:09.937614 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:09.938061 (+ 447us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:09.939386 (+ 1325us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:09.939442 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:09.943809 (+ 4367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:09.947909 (+ 4100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:09.947921 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:09.947947 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:09.948760 (+ 813us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:09.949390 (+ 630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:09.949403 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:09.949410 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:09.949864 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:09.950733 (+ 869us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:09.950745 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:09.951363 (+ 618us) client_negotiation.cc:769] Sending connection context
0807 18:14:09.951825 (+ 462us) client_negotiation.cc:240] Negotiation successful
0807 18:14:09.952561 (+ 736us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"thread_start_us":447,"threads_started":1}
I20260807 18:14:09.974537 20718 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.2:35923
I20260807 18:14:09.974589 20903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.2:35923 every 8 connection(s)
I20260807 18:14:09.976971 20718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
I20260807 18:14:09.980050 20907 heartbeater.cc:347] Connected to a master server at 127.17.110.62:44799
I20260807 18:14:09.980415 20907 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:09.981487 20907 heartbeater.cc:511] Master 127.17.110.62:44799 requested a full tablet report, sending...
I20260807 18:14:09.982079 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 20718
I20260807 18:14:09.982252 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance
I20260807 18:14:09.982982 17848 external_mini_cluster.cc:1428] Setting key aba50fe0d8867cb6b882222e30b99efd
I20260807 18:14:09.985167 20409 ts_manager.cc:194] Registered new tserver with Master: aa235ba635bf4b5ca63e16ce4aabed61 (127.17.110.2:35923)
I20260807 18:14:09.986346 20409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.2@KRBTEST.COM'} at 127.17.110.2:45071
I20260807 18:14:10.000480 20752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:09.974321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:35139 (local address 127.17.110.62:44799)
0807 18:14:09.974570 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:09.974577 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:09.978151 (+ 3574us) server_negotiation.cc:407] Connection header received
0807 18:14:09.978495 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:09.978502 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:09.978631 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:09.978805 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:09.980008 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:09.981289 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:09.983631 (+ 2342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:09.984183 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:09.984345 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:09.992789 (+ 8444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:09.992800 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:09.992809 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:09.992889 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:09.996635 (+ 3746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:09.997860 (+ 1225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:09.997866 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:09.997868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:09.998019 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:09.998632 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:09.998639 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:09.998642 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:09.999161 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:09.999317 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:09.999934 (+ 617us) server_negotiation.cc:299] Negotiation successful
0807 18:14:10.000223 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 18:14:10.000847 20906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:09.974501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:44799 (local address 127.17.110.2:35139)
0807 18:14:09.977901 (+ 3400us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:09.977933 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:09.978310 (+ 377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:09.978842 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:09.978847 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:09.979522 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:09.979807 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:09.979847 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:09.981519 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:09.981524 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:09.983336 (+ 1812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:09.983355 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:09.984688 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:09.984695 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:09.984891 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:09.990994 (+ 6103us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:09.991028 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:09.992355 (+ 1327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:09.996873 (+ 4518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:09.996891 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:09.996898 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:09.997622 (+ 724us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:09.998161 (+ 539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:09.998169 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:09.998175 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:09.998377 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:09.999317 (+ 940us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:09.999325 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:09.999598 (+ 273us) client_negotiation.cc:769] Sending connection context
0807 18:14:09.999796 (+ 198us) client_negotiation.cc:240] Negotiation successful
0807 18:14:10.000239 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3108,"thread_start_us":2930,"threads_started":1}
WARNING: no policy specified for kudu/127.17.110.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
I20260807 18:14:10.051344 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.3:0
--local_ip_for_outbound_sockets=127.17.110.3
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:44799
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:39545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:14:10.395355 20712 heartbeater.cc:503] Master 127.17.110.62:44799 was elected leader, sending a full tablet report...
W20260807 18:14:10.448825 20915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:10.449144 20915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:10.449335 20915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:10.456964 20915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:10.457093 20915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:10.457345 20915 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 18:14:10.457438 20915 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:14:10.457566 20915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.3
I20260807 18:14:10.468451 20915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:39545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.110.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.110.62:44799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.3
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:10.469848 20915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:10.471343 20915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:10.487349 20921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:10.487322 20922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:10.487285 20925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:10.488361 20915 server_base.cc:1034] running on GCE node
I20260807 18:14:10.489187 20915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:10.490351 20915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:10.491637 20915 hybrid_clock.cc:648] HybridClock initialized: now 1786126450491592 us; error 67 us; skew 500 ppm
I20260807 18:14:10.492093 20915 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:10 dist-test-slave-39l8 krb5kdc[20345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126450, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:10.496347 20915 init.cc:377] Logged in from keytab as kudu/127.17.110.3@KRBTEST.COM (short username kudu)
I20260807 18:14:10.498471 20915 webserver.cc:460] Webserver started at http://127.17.110.3:43649/ using document root <none> and password file <none>
I20260807 18:14:10.499398 20915 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:10.499585 20915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:10.499907 20915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:10.503299 20915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-2/data/instance:
uuid: "bad08fe563c74ec9bc51bf3159a4ac74"
format_stamp: "Formatted at 2026-08-07 18:14:10 on dist-test-slave-39l8"
server_key: "3f51a3c47b15d1eb208cded02d6d2717"
server_key_iv: "7cdaa5db101b3e8b1a2ed0d4820d3599"
server_key_version: "encryptionkey@0"
I20260807 18:14:10.504103 20915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance:
uuid: "bad08fe563c74ec9bc51bf3159a4ac74"
format_stamp: "Formatted at 2026-08-07 18:14:10 on dist-test-slave-39l8"
server_key: "3f51a3c47b15d1eb208cded02d6d2717"
server_key_iv: "7cdaa5db101b3e8b1a2ed0d4820d3599"
server_key_version: "encryptionkey@0"
I20260807 18:14:10.511061 20915 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260807 18:14:10.516083 20933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:10.518447 20915 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:14:10.518747 20915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-2/wal
uuid: "bad08fe563c74ec9bc51bf3159a4ac74"
format_stamp: "Formatted at 2026-08-07 18:14:10 on dist-test-slave-39l8"
server_key: "3f51a3c47b15d1eb208cded02d6d2717"
server_key_iv: "7cdaa5db101b3e8b1a2ed0d4820d3599"
server_key_version: "encryptionkey@0"
I20260807 18:14:10.519026 20915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:10.543742 20915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:10.549608 20915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:10.549981 20915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:10.554262 20915 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:14:10.554446 20915 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:10.554570 20915 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:14:10.554674 20915 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:14:10 dist-test-slave-39l8 krb5kdc[20345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126450, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:10.585530 20946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:10.554671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:46885 (local address 127.17.110.62:44799)
0807 18:14:10.555317 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:10.555326 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:10.557299 (+ 1973us) server_negotiation.cc:407] Connection header received
0807 18:14:10.559900 (+ 2601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:10.559908 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:10.560021 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:10.560164 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:10.563483 (+ 3319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:10.564965 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:10.567903 (+ 2938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:10.568480 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:10.568656 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:10.575526 (+ 6870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:10.575537 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:10.575544 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:10.575615 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:10.580143 (+ 4528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:10.581602 (+ 1459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:10.581610 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:10.581614 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:10.581786 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:10.582361 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:10.582365 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:10.582368 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:10.582776 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:10.583294 (+ 518us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:10.584785 (+ 1491us) server_negotiation.cc:299] Negotiation successful
0807 18:14:10.585069 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":310,"threads_started":1}
I20260807 18:14:10.586758 20949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:10.555741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:44799 (local address 127.17.110.3:46885)
0807 18:14:10.557070 (+ 1329us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:10.557126 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:10.559526 (+ 2400us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:10.560453 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:10.560463 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:10.561719 (+ 1256us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:10.563070 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:10.563112 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:10.565230 (+ 2118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:10.565240 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:10.567610 (+ 2370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:10.567657 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:10.568833 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:10.568847 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:10.569253 (+ 406us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:10.570244 (+ 991us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:10.570310 (+ 66us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:10.575110 (+ 4800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:10.580438 (+ 5328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:10.580450 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:10.580477 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:10.581418 (+ 941us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:10.581927 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:10.581933 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:10.581937 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:10.582199 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:10.583411 (+ 1212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:10.583423 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:10.584187 (+ 764us) client_negotiation.cc:769] Sending connection context
0807 18:14:10.584656 (+ 469us) client_negotiation.cc:240] Negotiation successful
0807 18:14:10.585316 (+ 660us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1018,"thread_start_us":539,"threads_started":1}
I20260807 18:14:10.617478 20946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:10.593780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:40705 (local address 127.17.110.62:44799)
0807 18:14:10.594011 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:10.594016 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:10.595577 (+ 1561us) server_negotiation.cc:407] Connection header received
0807 18:14:10.595879 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:10.595884 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:10.596000 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:10.596124 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:10.598335 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:10.599719 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:10.603049 (+ 3330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:10.603883 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:10.604077 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:10.607241 (+ 3164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:10.607252 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:10.607260 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:10.607320 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:10.612256 (+ 4936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:10.614030 (+ 1774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:10.614045 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:10.614050 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:10.614457 (+ 407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:10.615290 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:10.615296 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:10.615299 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:10.615746 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:10.615943 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:10.616660 (+ 717us) server_negotiation.cc:299] Negotiation successful
0807 18:14:10.617023 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 18:14:10.617628 21030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:10.593762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:44799 (local address 127.17.110.3:40705)
0807 18:14:10.595400 (+ 1638us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:10.595430 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:10.595673 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:10.596386 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:10.596394 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:10.597399 (+ 1005us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:10.598050 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:10.598076 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:10.600043 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:10.600054 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:10.602769 (+ 2715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:10.602804 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:10.604210 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:10.604221 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:10.604509 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:10.605429 (+ 920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:10.605460 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:10.606922 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:10.612571 (+ 5649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:10.612589 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:10.612593 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:10.613704 (+ 1111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:10.614765 (+ 1061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:10.614776 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:10.614783 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:10.615044 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:10.615910 (+ 866us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:10.615915 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:10.616235 (+ 320us) client_negotiation.cc:769] Sending connection context
0807 18:14:10.616465 (+ 230us) client_negotiation.cc:240] Negotiation successful
0807 18:14:10.616931 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1394,"thread_start_us":1192,"threads_started":1}
I20260807 18:14:10.618496 20915 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.3:40729
I20260807 18:14:10.618510 21098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.3:40729 every 8 connection(s)
I20260807 18:14:10.620184 20915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
I20260807 18:14:10.623346 21100 heartbeater.cc:347] Connected to a master server at 127.17.110.62:44799
I20260807 18:14:10.623653 21100 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:10.624583 21100 heartbeater.cc:511] Master 127.17.110.62:44799 requested a full tablet report, sending...
I20260807 18:14:10.626361 20409 ts_manager.cc:194] Registered new tserver with Master: bad08fe563c74ec9bc51bf3159a4ac74 (127.17.110.3:40729)
I20260807 18:14:10.627789 20409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.3@KRBTEST.COM'} at 127.17.110.3:46885
I20260807 18:14:10.629649 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 20915
I20260807 18:14:10.629839 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance
I20260807 18:14:10.630405 17848 external_mini_cluster.cc:1428] Setting key 157b89ee513ffbc10aa6f4fa07470d3d
I20260807 18:14:10.634856 17848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:14:10 dist-test-slave-39l8 krb5kdc[20345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 18:14:10 dist-test-slave-39l8 krb5kdc[20345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:10.697745 20946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:10.679371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42440 (local address 127.17.110.62:44799)
0807 18:14:10.679599 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:10.679607 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:10.679742 (+ 135us) server_negotiation.cc:407] Connection header received
0807 18:14:10.679960 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:10.679966 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:10.680173 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:10.680361 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:10.682046 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:10.682928 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:10.684535 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:10.685036 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:10.685164 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:10.690168 (+ 5004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:10.690174 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:10.690177 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:10.690226 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:10.693668 (+ 3442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:10.694909 (+ 1241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:10.694915 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:10.694918 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:10.695095 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:10.696014 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:10.696020 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:10.696022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:10.696587 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:10.696896 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:10.697261 (+ 365us) server_negotiation.cc:299] Negotiation successful
0807 18:14:10.697467 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 18:14:10.703229 20409 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42440:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:14:10.710902 20409 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:14:10.755362 21128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:10.740879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59840 (local address 127.17.110.2:35923)
0807 18:14:10.745683 (+ 4804us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:10.745691 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:10.745721 (+ 30us) server_negotiation.cc:407] Connection header received
0807 18:14:10.745816 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:10.745827 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:10.746045 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:10.746267 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:14:10.747282 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:10.748393 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:10.752745 (+ 4352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:10.753817 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:10.754015 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:10.754318 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:10.754749 (+ 431us) server_negotiation.cc:299] Negotiation successful
0807 18:14:10.754969 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4623,"thread_start_us":4475,"threads_started":1}
I20260807 18:14:10.755486 21122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:10.740559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:35923 (local address 127.0.0.1:59840)
0807 18:14:10.745067 (+ 4508us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:10.745083 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:10.745172 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:10.746512 (+ 1340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:10.746516 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:10.746558 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:14:10.747056 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:10.747069 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:10.748558 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:10.748565 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:10.752462 (+ 3897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:10.752474 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:10.754082 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:10.754090 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:10.754360 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:10.754439 (+ 79us) client_negotiation.cc:769] Sending connection context
0807 18:14:10.754646 (+ 207us) client_negotiation.cc:240] Negotiation successful
0807 18:14:10.754984 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4376,"mutex_wait_us":482,"thread_start_us":4003,"threads_started":1}
I20260807 18:14:10.756016 21121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:10.726578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:40729 (local address 127.0.0.1:56962)
0807 18:14:10.741250 (+ 14672us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:10.741342 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:10.741734 (+ 392us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:10.745739 (+ 4005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:10.745747 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:10.745796 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:14:10.746314 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:10.746339 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:10.748699 (+ 2360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:10.748706 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:10.752416 (+ 3710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:10.752437 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:10.754773 (+ 2336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:10.754779 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:10.755033 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:10.755062 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 18:14:10.755155 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 18:14:10.755296 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12024,"mutex_wait_us":99,"thread_start_us":12212,"threads_started":1}
I20260807 18:14:10.756173 21125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:10.726851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56962 (local address 127.17.110.3:40729)
0807 18:14:10.744564 (+ 17713us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:10.744579 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:10.744633 (+ 54us) server_negotiation.cc:407] Connection header received
0807 18:14:10.744815 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:10.744827 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:10.745254 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:10.745520 (+ 266us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:14:10.746620 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:10.748507 (+ 1887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:10.752654 (+ 4147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:10.754192 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:10.754327 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:10.754580 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:10.755608 (+ 1028us) server_negotiation.cc:299] Negotiation successful
0807 18:14:10.755822 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17503,"thread_start_us":17271,"threads_started":1}
I20260807 18:14:10.756904 21127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:10.726763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:35891 (local address 127.0.0.1:56934)
0807 18:14:10.744924 (+ 18161us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:10.744950 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:10.745131 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:10.745942 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:10.745946 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:10.745966 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:14:10.746439 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:10.746454 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:10.748753 (+ 2299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:10.748760 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:10.752074 (+ 3314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:10.752093 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:10.754552 (+ 2459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:10.754562 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:10.754793 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:10.754829 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 18:14:10.754930 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 18:14:10.756539 (+ 1609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17986,"mutex_wait_us":1788,"thread_start_us":16540,"threads_started":1}
I20260807 18:14:10.757341 21126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:10.727718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56934 (local address 127.17.110.1:35891)
0807 18:14:10.742968 (+ 15250us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:10.742982 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:10.745295 (+ 2313us) server_negotiation.cc:407] Connection header received
0807 18:14:10.745426 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:10.745433 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:10.745675 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:10.745876 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:14:10.746674 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:10.748510 (+ 1836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:10.752295 (+ 3785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:10.753935 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:10.754118 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:10.754361 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:10.756586 (+ 2225us) server_negotiation.cc:299] Negotiation successful
0807 18:14:10.756795 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14994,"thread_start_us":15259,"threads_started":1}
I20260807 18:14:10.760922 20813 tablet_service.cc:1467] Processing CreateTablet for tablet f5dc56a2048e43d29600bb2df2504c67 (DEFAULT_TABLE table=test-table [id=81d0e2f74a26477280894bdbf849d739]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:14:10.760923 21003 tablet_service.cc:1467] Processing CreateTablet for tablet f5dc56a2048e43d29600bb2df2504c67 (DEFAULT_TABLE table=test-table [id=81d0e2f74a26477280894bdbf849d739]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:14:10.761059 20605 tablet_service.cc:1467] Processing CreateTablet for tablet f5dc56a2048e43d29600bb2df2504c67 (DEFAULT_TABLE table=test-table [id=81d0e2f74a26477280894bdbf849d739]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:14:10.762918 21003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5dc56a2048e43d29600bb2df2504c67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:10.762920 20813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5dc56a2048e43d29600bb2df2504c67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:10.763726 20605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5dc56a2048e43d29600bb2df2504c67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:10.777283 21130 tablet_bootstrap.cc:492] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20: Bootstrap starting.
I20260807 18:14:10.780608 21134 tablet_bootstrap.cc:492] T f5dc56a2048e43d29600bb2df2504c67 P aa235ba635bf4b5ca63e16ce4aabed61: Bootstrap starting.
I20260807 18:14:10.781796 21130 tablet_bootstrap.cc:654] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:10.783782 21138 tablet_bootstrap.cc:492] T f5dc56a2048e43d29600bb2df2504c67 P bad08fe563c74ec9bc51bf3159a4ac74: Bootstrap starting.
I20260807 18:14:10.783726 21130 log.cc:824] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20: Log is configured to *not* fsync() on all Append() calls
I20260807 18:14:10.785286 21134 tablet_bootstrap.cc:654] T f5dc56a2048e43d29600bb2df2504c67 P aa235ba635bf4b5ca63e16ce4aabed61: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:10.787820 21138 tablet_bootstrap.cc:654] T f5dc56a2048e43d29600bb2df2504c67 P bad08fe563c74ec9bc51bf3159a4ac74: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:10.787873 21134 log.cc:824] T f5dc56a2048e43d29600bb2df2504c67 P aa235ba635bf4b5ca63e16ce4aabed61: Log is configured to *not* fsync() on all Append() calls
I20260807 18:14:10.789310 21138 log.cc:824] T f5dc56a2048e43d29600bb2df2504c67 P bad08fe563c74ec9bc51bf3159a4ac74: Log is configured to *not* fsync() on all Append() calls
I20260807 18:14:10.792188 21134 tablet_bootstrap.cc:492] T f5dc56a2048e43d29600bb2df2504c67 P aa235ba635bf4b5ca63e16ce4aabed61: No bootstrap required, opened a new log
I20260807 18:14:10.792186 21130 tablet_bootstrap.cc:492] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20: No bootstrap required, opened a new log
I20260807 18:14:10.792307 21138 tablet_bootstrap.cc:492] T f5dc56a2048e43d29600bb2df2504c67 P bad08fe563c74ec9bc51bf3159a4ac74: No bootstrap required, opened a new log
I20260807 18:14:10.792549 21130 ts_tablet_manager.cc:1397] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20: Time spent bootstrapping tablet: real 0.016s user 0.013s sys 0.000s
I20260807 18:14:10.792550 21134 ts_tablet_manager.cc:1397] T f5dc56a2048e43d29600bb2df2504c67 P aa235ba635bf4b5ca63e16ce4aabed61: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.001s
I20260807 18:14:10.792677 21138 ts_tablet_manager.cc:1397] T f5dc56a2048e43d29600bb2df2504c67 P bad08fe563c74ec9bc51bf3159a4ac74: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 18:14:10.801805 21138 raft_consensus.cc:348] T f5dc56a2048e43d29600bb2df2504c67 P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } } peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } }
I20260807 18:14:10.802299 21138 raft_consensus.cc:374] T f5dc56a2048e43d29600bb2df2504c67 P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:10.802408 21138 raft_consensus.cc:729] T f5dc56a2048e43d29600bb2df2504c67 P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad08fe563c74ec9bc51bf3159a4ac74, State: Initialized, Role: FOLLOWER
I20260807 18:14:10.803372 21138 consensus_queue.cc:260] T f5dc56a2048e43d29600bb2df2504c67 P bad08fe563c74ec9bc51bf3159a4ac74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } } peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } }
I20260807 18:14:10.804813 21100 heartbeater.cc:503] Master 127.17.110.62:44799 was elected leader, sending a full tablet report...
I20260807 18:14:10.805405 21138 ts_tablet_manager.cc:1428] T f5dc56a2048e43d29600bb2df2504c67 P bad08fe563c74ec9bc51bf3159a4ac74: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260807 18:14:10.805590 21134 raft_consensus.cc:348] T f5dc56a2048e43d29600bb2df2504c67 P aa235ba635bf4b5ca63e16ce4aabed61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } } peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } }
I20260807 18:14:10.806151 21134 raft_consensus.cc:374] T f5dc56a2048e43d29600bb2df2504c67 P aa235ba635bf4b5ca63e16ce4aabed61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:10.806263 21134 raft_consensus.cc:729] T f5dc56a2048e43d29600bb2df2504c67 P aa235ba635bf4b5ca63e16ce4aabed61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa235ba635bf4b5ca63e16ce4aabed61, State: Initialized, Role: FOLLOWER
I20260807 18:14:10.807267 21134 consensus_queue.cc:260] T f5dc56a2048e43d29600bb2df2504c67 P aa235ba635bf4b5ca63e16ce4aabed61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } } peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } }
I20260807 18:14:10.807760 21130 raft_consensus.cc:348] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } } peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } }
I20260807 18:14:10.808418 21130 raft_consensus.cc:374] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:10.808528 21130 raft_consensus.cc:729] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 067119311361489a9621056d34ddce20, State: Initialized, Role: FOLLOWER
I20260807 18:14:10.808805 20907 heartbeater.cc:503] Master 127.17.110.62:44799 was elected leader, sending a full tablet report...
I20260807 18:14:10.809197 21130 consensus_queue.cc:260] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } } peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } }
I20260807 18:14:10.809459 21134 ts_tablet_manager.cc:1428] T f5dc56a2048e43d29600bb2df2504c67 P aa235ba635bf4b5ca63e16ce4aabed61: Time spent starting tablet: real 0.017s user 0.014s sys 0.003s
I20260807 18:14:10.812429 21130 ts_tablet_manager.cc:1428] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20: Time spent starting tablet: real 0.020s user 0.015s sys 0.003s
W20260807 18:14:10.873186 21102 tablet.cc:2369] T f5dc56a2048e43d29600bb2df2504c67 P bad08fe563c74ec9bc51bf3159a4ac74: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:14:10.885504 20714 tablet.cc:2369] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:14:10.980321 20908 tablet.cc:2369] T f5dc56a2048e43d29600bb2df2504c67 P aa235ba635bf4b5ca63e16ce4aabed61: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:14:11.007185 21147 raft_consensus.cc:482] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:14:11.007618 21147 raft_consensus.cc:504] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } } peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } }
I20260807 18:14:11.009490 21147 leader_election.cc:288] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad08fe563c74ec9bc51bf3159a4ac74 (127.17.110.3:40729), aa235ba635bf4b5ca63e16ce4aabed61 (127.17.110.2:35923)
I20260807 18:14:11.019704 21125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:11.010329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:54565 (local address 127.17.110.3:40729)
0807 18:14:11.010572 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:11.010579 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:11.011263 (+ 684us) server_negotiation.cc:407] Connection header received
0807 18:14:11.011575 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:11.011583 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:11.011757 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:11.011914 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:14:11.012711 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.013849 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.016771 (+ 2922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.018131 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.018342 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:11.018426 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:11.019291 (+ 865us) server_negotiation.cc:299] Negotiation successful
0807 18:14:11.019439 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 18:14:11.019716 21128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:11.011660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:42319 (local address 127.17.110.2:35923)
0807 18:14:11.011900 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:11.011909 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:11.012335 (+ 426us) server_negotiation.cc:407] Connection header received
0807 18:14:11.012504 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:11.012508 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:11.012569 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:11.012685 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:14:11.013212 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.014216 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.016771 (+ 2555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.018131 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.018342 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:11.018435 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:11.019250 (+ 815us) server_negotiation.cc:299] Negotiation successful
0807 18:14:11.019412 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 18:14:11.020033 21148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:11.010217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:40729 (local address 127.17.110.1:54565)
0807 18:14:11.011116 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:11.011136 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:11.011346 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:11.012043 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:11.012047 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:11.012093 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:14:11.012551 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:11.012562 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.014004 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.014008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:11.016513 (+ 2505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:11.016534 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.018516 (+ 1982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.018524 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:11.018920 (+ 396us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:11.019052 (+ 132us) client_negotiation.cc:769] Sending connection context
0807 18:14:11.019229 (+ 177us) client_negotiation.cc:240] Negotiation successful
0807 18:14:11.019372 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"mutex_wait_us":135,"thread_start_us":549,"threads_started":1}
I20260807 18:14:11.020083 21149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:11.011390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:35923 (local address 127.17.110.1:42319)
0807 18:14:11.012200 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:11.012223 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:11.012359 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:11.012713 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:11.012722 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:11.012744 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:14:11.013023 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:11.013030 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.014373 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.014378 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:11.016537 (+ 2159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:11.016546 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.018529 (+ 1983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.018533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:11.018924 (+ 391us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:11.019052 (+ 128us) client_negotiation.cc:769] Sending connection context
0807 18:14:11.019177 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 18:14:11.019372 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"mutex_wait_us":125,"thread_start_us":433,"threads_started":1}
I20260807 18:14:11.021523 21036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5dc56a2048e43d29600bb2df2504c67" candidate_uuid: "067119311361489a9621056d34ddce20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" is_pre_election: true
I20260807 18:14:11.021514 20840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5dc56a2048e43d29600bb2df2504c67" candidate_uuid: "067119311361489a9621056d34ddce20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" is_pre_election: true
I20260807 18:14:11.022122 20840 raft_consensus.cc:2454] T f5dc56a2048e43d29600bb2df2504c67 P aa235ba635bf4b5ca63e16ce4aabed61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 067119311361489a9621056d34ddce20 in term 0.
I20260807 18:14:11.022123 21036 raft_consensus.cc:2454] T f5dc56a2048e43d29600bb2df2504c67 P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 067119311361489a9621056d34ddce20 in term 0.
I20260807 18:14:11.022996 20536 leader_election.cc:302] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 067119311361489a9621056d34ddce20, aa235ba635bf4b5ca63e16ce4aabed61; no voters:
I20260807 18:14:11.023504 21147 raft_consensus.cc:2781] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:14:11.023700 21147 raft_consensus.cc:482] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:14:11.023914 21147 raft_consensus.cc:3037] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:11.027112 21147 raft_consensus.cc:504] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } } peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } }
I20260807 18:14:11.027907 21147 leader_election.cc:288] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20 [CANDIDATE]: Term 1 election: Requested vote from peers bad08fe563c74ec9bc51bf3159a4ac74 (127.17.110.3:40729), aa235ba635bf4b5ca63e16ce4aabed61 (127.17.110.2:35923)
I20260807 18:14:11.028676 20840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5dc56a2048e43d29600bb2df2504c67" candidate_uuid: "067119311361489a9621056d34ddce20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa235ba635bf4b5ca63e16ce4aabed61"
I20260807 18:14:11.028672 21036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5dc56a2048e43d29600bb2df2504c67" candidate_uuid: "067119311361489a9621056d34ddce20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad08fe563c74ec9bc51bf3159a4ac74"
I20260807 18:14:11.028928 21036 raft_consensus.cc:3037] T f5dc56a2048e43d29600bb2df2504c67 P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:11.028923 20840 raft_consensus.cc:3037] T f5dc56a2048e43d29600bb2df2504c67 P aa235ba635bf4b5ca63e16ce4aabed61 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:11.031502 21036 raft_consensus.cc:2454] T f5dc56a2048e43d29600bb2df2504c67 P bad08fe563c74ec9bc51bf3159a4ac74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 067119311361489a9621056d34ddce20 in term 1.
I20260807 18:14:11.031512 20840 raft_consensus.cc:2454] T f5dc56a2048e43d29600bb2df2504c67 P aa235ba635bf4b5ca63e16ce4aabed61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 067119311361489a9621056d34ddce20 in term 1.
I20260807 18:14:11.032191 20536 leader_election.cc:302] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 067119311361489a9621056d34ddce20, bad08fe563c74ec9bc51bf3159a4ac74; no voters:
I20260807 18:14:11.032562 21147 raft_consensus.cc:2781] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:14:11.033334 21147 raft_consensus.cc:686] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20 [term 1 LEADER]: Becoming Leader. State: Replica: 067119311361489a9621056d34ddce20, State: Running, Role: LEADER
I20260807 18:14:11.033903 21147 consensus_queue.cc:237] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } } peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } }
I20260807 18:14:11.039345 20408 catalog_manager.cc:5495] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20 reported cstate change: term changed from 0 to 1, leader changed from <none> to 067119311361489a9621056d34ddce20 (127.17.110.1). New cstate: current_term: 1 leader_uuid: "067119311361489a9621056d34ddce20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } health_report { overall_health: UNKNOWN } } }
Aug 07 18:14:11 dist-test-slave-39l8 krb5kdc[20345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126448, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:11.096242 21159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:11.078511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:44799 (local address 127.0.0.1:42442)
0807 18:14:11.079223 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:11.079241 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:11.079442 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:11.079794 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:11.079799 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:11.080961 (+ 1162us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:11.081319 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:11.081331 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.082479 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.082486 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:11.083607 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:11.083622 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.084432 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.084439 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:11.084603 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:11.085444 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:11.085466 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:11.088426 (+ 2960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:11.092863 (+ 4437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:11.092884 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:11.092892 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:11.093470 (+ 578us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:11.094155 (+ 685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:11.094160 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:11.094162 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:11.094339 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:11.095162 (+ 823us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:11.095169 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:11.095434 (+ 265us) client_negotiation.cc:769] Sending connection context
0807 18:14:11.095600 (+ 166us) client_negotiation.cc:240] Negotiation successful
0807 18:14:11.095882 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":411,"threads_started":1}
I20260807 18:14:11.096745 20946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:11.078444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42442 (local address 127.17.110.62:44799)
0807 18:14:11.078652 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:11.078660 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:11.079346 (+ 686us) server_negotiation.cc:407] Connection header received
0807 18:14:11.079567 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:11.079573 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:11.079648 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:11.079758 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:11.081527 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.082324 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.083772 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.084227 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.084385 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:11.088788 (+ 4403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:11.088795 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:11.088799 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:11.088842 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:11.092642 (+ 3800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:11.093673 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:11.093678 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:11.093680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:11.093831 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:11.094547 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:11.094552 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:11.094554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:11.094987 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:11.095130 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:11.095777 (+ 647us) server_negotiation.cc:299] Negotiation successful
0807 18:14:11.096169 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":144,"server-negotiator.queue_time_us":66}
I20260807 18:14:11.104317 20408 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:42442:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 18:14:11.105806 20408 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:14:11.115461 20813 tablet_service.cc:1467] Processing CreateTablet for tablet 6208957d42d64739af9e9a9de8fa915f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5376dd95c9314fc39da14da7437ca65a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:14:11.115370 20605 tablet_service.cc:1467] Processing CreateTablet for tablet 6208957d42d64739af9e9a9de8fa915f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5376dd95c9314fc39da14da7437ca65a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:14:11.116340 20813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6208957d42d64739af9e9a9de8fa915f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:11.115842 21003 tablet_service.cc:1467] Processing CreateTablet for tablet 6208957d42d64739af9e9a9de8fa915f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5376dd95c9314fc39da14da7437ca65a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:14:11.116465 20605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6208957d42d64739af9e9a9de8fa915f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:11.116708 21003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6208957d42d64739af9e9a9de8fa915f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:11.124677 21134 tablet_bootstrap.cc:492] T 6208957d42d64739af9e9a9de8fa915f P aa235ba635bf4b5ca63e16ce4aabed61: Bootstrap starting.
I20260807 18:14:11.124734 21130 tablet_bootstrap.cc:492] T 6208957d42d64739af9e9a9de8fa915f P 067119311361489a9621056d34ddce20: Bootstrap starting.
I20260807 18:14:11.126400 21138 tablet_bootstrap.cc:492] T 6208957d42d64739af9e9a9de8fa915f P bad08fe563c74ec9bc51bf3159a4ac74: Bootstrap starting.
I20260807 18:14:11.126737 21130 tablet_bootstrap.cc:654] T 6208957d42d64739af9e9a9de8fa915f P 067119311361489a9621056d34ddce20: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:11.127122 21134 tablet_bootstrap.cc:654] T 6208957d42d64739af9e9a9de8fa915f P aa235ba635bf4b5ca63e16ce4aabed61: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:11.128073 21138 tablet_bootstrap.cc:654] T 6208957d42d64739af9e9a9de8fa915f P bad08fe563c74ec9bc51bf3159a4ac74: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:11.130208 21130 tablet_bootstrap.cc:492] T 6208957d42d64739af9e9a9de8fa915f P 067119311361489a9621056d34ddce20: No bootstrap required, opened a new log
I20260807 18:14:11.130406 21130 ts_tablet_manager.cc:1397] T 6208957d42d64739af9e9a9de8fa915f P 067119311361489a9621056d34ddce20: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260807 18:14:11.130923 21134 tablet_bootstrap.cc:492] T 6208957d42d64739af9e9a9de8fa915f P aa235ba635bf4b5ca63e16ce4aabed61: No bootstrap required, opened a new log
I20260807 18:14:11.131093 21134 ts_tablet_manager.cc:1397] T 6208957d42d64739af9e9a9de8fa915f P aa235ba635bf4b5ca63e16ce4aabed61: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.003s
I20260807 18:14:11.131095 21138 tablet_bootstrap.cc:492] T 6208957d42d64739af9e9a9de8fa915f P bad08fe563c74ec9bc51bf3159a4ac74: No bootstrap required, opened a new log
I20260807 18:14:11.131276 21138 ts_tablet_manager.cc:1397] T 6208957d42d64739af9e9a9de8fa915f P bad08fe563c74ec9bc51bf3159a4ac74: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 18:14:11.131423 21130 raft_consensus.cc:348] T 6208957d42d64739af9e9a9de8fa915f P 067119311361489a9621056d34ddce20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:11.131701 21130 raft_consensus.cc:374] T 6208957d42d64739af9e9a9de8fa915f P 067119311361489a9621056d34ddce20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:11.131839 21130 raft_consensus.cc:729] T 6208957d42d64739af9e9a9de8fa915f P 067119311361489a9621056d34ddce20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 067119311361489a9621056d34ddce20, State: Initialized, Role: FOLLOWER
I20260807 18:14:11.131973 21134 raft_consensus.cc:348] T 6208957d42d64739af9e9a9de8fa915f P aa235ba635bf4b5ca63e16ce4aabed61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:11.132277 21134 raft_consensus.cc:374] T 6208957d42d64739af9e9a9de8fa915f P aa235ba635bf4b5ca63e16ce4aabed61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:11.132136 21130 consensus_queue.cc:260] T 6208957d42d64739af9e9a9de8fa915f P 067119311361489a9621056d34ddce20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:11.132385 21134 raft_consensus.cc:729] T 6208957d42d64739af9e9a9de8fa915f P aa235ba635bf4b5ca63e16ce4aabed61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa235ba635bf4b5ca63e16ce4aabed61, State: Initialized, Role: FOLLOWER
I20260807 18:14:11.132630 21134 consensus_queue.cc:260] T 6208957d42d64739af9e9a9de8fa915f P aa235ba635bf4b5ca63e16ce4aabed61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:11.132655 21147 tablet_replica.cc:439] T 6208957d42d64739af9e9a9de8fa915f P 067119311361489a9621056d34ddce20: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } } }
I20260807 18:14:11.132869 21130 ts_tablet_manager.cc:1428] T 6208957d42d64739af9e9a9de8fa915f P 067119311361489a9621056d34ddce20: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 18:14:11.132997 21147 tablet_replica.cc:442] T 6208957d42d64739af9e9a9de8fa915f P 067119311361489a9621056d34ddce20: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:14:11.133258 21134 ts_tablet_manager.cc:1428] T 6208957d42d64739af9e9a9de8fa915f P aa235ba635bf4b5ca63e16ce4aabed61: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 18:14:11.132232 21138 raft_consensus.cc:348] T 6208957d42d64739af9e9a9de8fa915f P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:11.133876 21138 raft_consensus.cc:374] T 6208957d42d64739af9e9a9de8fa915f P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:11.134013 21138 raft_consensus.cc:729] T 6208957d42d64739af9e9a9de8fa915f P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad08fe563c74ec9bc51bf3159a4ac74, State: Initialized, Role: FOLLOWER
I20260807 18:14:11.134002 21146 tablet_replica.cc:439] T 6208957d42d64739af9e9a9de8fa915f P aa235ba635bf4b5ca63e16ce4aabed61: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } } }
I20260807 18:14:11.134441 21146 tablet_replica.cc:442] T 6208957d42d64739af9e9a9de8fa915f P aa235ba635bf4b5ca63e16ce4aabed61: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:14:11.134581 21138 consensus_queue.cc:260] T 6208957d42d64739af9e9a9de8fa915f P bad08fe563c74ec9bc51bf3159a4ac74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:11.135118 21143 tablet_replica.cc:439] T 6208957d42d64739af9e9a9de8fa915f P bad08fe563c74ec9bc51bf3159a4ac74: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } } }
I20260807 18:14:11.135471 21143 tablet_replica.cc:442] T 6208957d42d64739af9e9a9de8fa915f P bad08fe563c74ec9bc51bf3159a4ac74: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:14:11.135895 21138 ts_tablet_manager.cc:1428] T 6208957d42d64739af9e9a9de8fa915f P bad08fe563c74ec9bc51bf3159a4ac74: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260807 18:14:11.325381 21143 raft_consensus.cc:482] T 6208957d42d64739af9e9a9de8fa915f P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:14:11.325664 21143 raft_consensus.cc:504] T 6208957d42d64739af9e9a9de8fa915f P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:11.327447 21143 leader_election.cc:288] T 6208957d42d64739af9e9a9de8fa915f P bad08fe563c74ec9bc51bf3159a4ac74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 067119311361489a9621056d34ddce20 (127.17.110.1:35891), aa235ba635bf4b5ca63e16ce4aabed61 (127.17.110.2:35923)
I20260807 18:14:11.334367 21128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:11.327981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:41705 (local address 127.17.110.2:35923)
0807 18:14:11.328295 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:11.328305 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:11.328994 (+ 689us) server_negotiation.cc:407] Connection header received
0807 18:14:11.329278 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:11.329283 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:11.329350 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:11.329461 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:14:11.329977 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.330806 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.332523 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.333343 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.333537 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:11.333643 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:11.333917 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 18:14:11.334062 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:14:11.334444 21164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:11.327737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:35923 (local address 127.17.110.3:41705)
0807 18:14:11.328850 (+ 1113us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:11.328874 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:11.329091 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:11.329485 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:11.329489 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:11.329526 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:14:11.329792 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:11.329802 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.330946 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.330953 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:11.332337 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:11.332351 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.333538 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.333544 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:11.333739 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:11.333766 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 18:14:11.333884 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 18:14:11.334036 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":960,"thread_start_us":1172,"threads_started":1}
I20260807 18:14:11.335237 20840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6208957d42d64739af9e9a9de8fa915f" candidate_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" is_pre_election: true
I20260807 18:14:11.335551 20840 raft_consensus.cc:2454] T 6208957d42d64739af9e9a9de8fa915f P aa235ba635bf4b5ca63e16ce4aabed61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bad08fe563c74ec9bc51bf3159a4ac74 in term 0.
I20260807 18:14:11.335597 21165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:11.327770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:35891 (local address 127.17.110.3:49763)
0807 18:14:11.329186 (+ 1416us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:11.329201 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:11.329368 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:11.329735 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:11.329739 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:11.329772 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:14:11.330165 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:11.330175 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.331757 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.331763 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:11.333175 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:11.333198 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.334697 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.334733 (+ 36us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:11.334930 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:11.334957 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 18:14:11.335087 (+ 130us) client_negotiation.cc:240] Negotiation successful
0807 18:14:11.335235 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1297,"thread_start_us":799,"threads_started":1}
I20260807 18:14:11.335597 21166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:11.327869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:49763 (local address 127.17.110.1:35891)
0807 18:14:11.328648 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:11.328658 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:11.329309 (+ 651us) server_negotiation.cc:407] Connection header received
0807 18:14:11.329518 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:11.329522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:11.329593 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:11.329713 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:14:11.330611 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.331587 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.333405 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.334244 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.334462 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:11.334554 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:11.335134 (+ 580us) server_negotiation.cc:299] Negotiation successful
0807 18:14:11.335289 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":466,"threads_started":1}
I20260807 18:14:11.336233 20939 leader_election.cc:302] T 6208957d42d64739af9e9a9de8fa915f P bad08fe563c74ec9bc51bf3159a4ac74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa235ba635bf4b5ca63e16ce4aabed61, bad08fe563c74ec9bc51bf3159a4ac74; no voters:
I20260807 18:14:11.336799 21143 raft_consensus.cc:2781] T 6208957d42d64739af9e9a9de8fa915f P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:14:11.336956 21143 raft_consensus.cc:482] T 6208957d42d64739af9e9a9de8fa915f P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:14:11.337054 21143 raft_consensus.cc:3037] T 6208957d42d64739af9e9a9de8fa915f P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:11.336915 20638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6208957d42d64739af9e9a9de8fa915f" candidate_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "067119311361489a9621056d34ddce20" is_pre_election: true
I20260807 18:14:11.337563 20638 raft_consensus.cc:2454] T 6208957d42d64739af9e9a9de8fa915f P 067119311361489a9621056d34ddce20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bad08fe563c74ec9bc51bf3159a4ac74 in term 0.
I20260807 18:14:11.340436 21143 raft_consensus.cc:504] T 6208957d42d64739af9e9a9de8fa915f P bad08fe563c74ec9bc51bf3159a4ac74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:11.341195 21143 leader_election.cc:288] T 6208957d42d64739af9e9a9de8fa915f P bad08fe563c74ec9bc51bf3159a4ac74 [CANDIDATE]: Term 1 election: Requested vote from peers 067119311361489a9621056d34ddce20 (127.17.110.1:35891), aa235ba635bf4b5ca63e16ce4aabed61 (127.17.110.2:35923)
I20260807 18:14:11.341892 20638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6208957d42d64739af9e9a9de8fa915f" candidate_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "067119311361489a9621056d34ddce20"
I20260807 18:14:11.341997 20840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6208957d42d64739af9e9a9de8fa915f" candidate_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa235ba635bf4b5ca63e16ce4aabed61"
I20260807 18:14:11.342151 20638 raft_consensus.cc:3037] T 6208957d42d64739af9e9a9de8fa915f P 067119311361489a9621056d34ddce20 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:11.342374 20840 raft_consensus.cc:3037] T 6208957d42d64739af9e9a9de8fa915f P aa235ba635bf4b5ca63e16ce4aabed61 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:11.344576 20840 raft_consensus.cc:2454] T 6208957d42d64739af9e9a9de8fa915f P aa235ba635bf4b5ca63e16ce4aabed61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bad08fe563c74ec9bc51bf3159a4ac74 in term 1.
I20260807 18:14:11.344722 20638 raft_consensus.cc:2454] T 6208957d42d64739af9e9a9de8fa915f P 067119311361489a9621056d34ddce20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bad08fe563c74ec9bc51bf3159a4ac74 in term 1.
I20260807 18:14:11.345170 20939 leader_election.cc:302] T 6208957d42d64739af9e9a9de8fa915f P bad08fe563c74ec9bc51bf3159a4ac74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa235ba635bf4b5ca63e16ce4aabed61, bad08fe563c74ec9bc51bf3159a4ac74; no voters:
I20260807 18:14:11.345536 21143 raft_consensus.cc:2781] T 6208957d42d64739af9e9a9de8fa915f P bad08fe563c74ec9bc51bf3159a4ac74 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:14:11.346083 21143 raft_consensus.cc:686] T 6208957d42d64739af9e9a9de8fa915f P bad08fe563c74ec9bc51bf3159a4ac74 [term 1 LEADER]: Becoming Leader. State: Replica: bad08fe563c74ec9bc51bf3159a4ac74, State: Running, Role: LEADER
I20260807 18:14:11.346678 21143 consensus_queue.cc:237] T 6208957d42d64739af9e9a9de8fa915f P bad08fe563c74ec9bc51bf3159a4ac74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:11.350665 21168 tablet_replica.cc:439] T 6208957d42d64739af9e9a9de8fa915f P bad08fe563c74ec9bc51bf3159a4ac74: TxnStatusTablet state changed. Reason: New leader bad08fe563c74ec9bc51bf3159a4ac74. Latest consensus state: current_term: 1 leader_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } } }
I20260807 18:14:11.351145 21168 tablet_replica.cc:442] T 6208957d42d64739af9e9a9de8fa915f P bad08fe563c74ec9bc51bf3159a4ac74: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:14:11.352289 21170 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 18:14:11.352610 21170 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 18:14:11.353081 20409 catalog_manager.cc:5495] T 6208957d42d64739af9e9a9de8fa915f P bad08fe563c74ec9bc51bf3159a4ac74 reported cstate change: term changed from 0 to 1, leader changed from <none> to bad08fe563c74ec9bc51bf3159a4ac74 (127.17.110.3). New cstate: current_term: 1 leader_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } health_report { overall_health: HEALTHY } } }
I20260807 18:14:11.383504 21159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:11.375280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:40729 (local address 127.0.0.1:36110)
0807 18:14:11.375696 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:11.375722 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:11.375914 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:11.376522 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:11.376527 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:11.376543 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:14:11.376781 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:11.376796 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.378046 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.378053 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:11.379926 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:11.379949 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.381228 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.381235 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:11.381453 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:11.381547 (+ 94us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:14:11.382853 (+ 1306us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:14:11.382889 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 18:14:11.382995 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 18:14:11.383188 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260807 18:14:11.383500 21125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:11.375415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36110 (local address 127.17.110.3:40729)
0807 18:14:11.375643 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:11.375649 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:11.375965 (+ 316us) server_negotiation.cc:407] Connection header received
0807 18:14:11.376218 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:11.376225 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:11.376317 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:11.376461 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:14:11.376970 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.377862 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.380168 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.380903 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.381055 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:11.381824 (+ 769us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:14:11.382638 (+ 814us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:14:11.382862 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:11.383054 (+ 192us) server_negotiation.cc:299] Negotiation successful
0807 18:14:11.383157 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 18:14:11.395474 20840 raft_consensus.cc:1261] T 6208957d42d64739af9e9a9de8fa915f P aa235ba635bf4b5ca63e16ce4aabed61 [term 1 FOLLOWER]: Refusing update from remote peer bad08fe563c74ec9bc51bf3159a4ac74: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:14:11.395710 20638 raft_consensus.cc:1261] T 6208957d42d64739af9e9a9de8fa915f P 067119311361489a9621056d34ddce20 [term 1 FOLLOWER]: Refusing update from remote peer bad08fe563c74ec9bc51bf3159a4ac74: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:14:11.396956 21168 consensus_queue.cc:1035] T 6208957d42d64739af9e9a9de8fa915f P bad08fe563c74ec9bc51bf3159a4ac74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:14:11.397451 21143 consensus_queue.cc:1035] T 6208957d42d64739af9e9a9de8fa915f P bad08fe563c74ec9bc51bf3159a4ac74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:14:11.412894 21146 tablet_replica.cc:439] T 6208957d42d64739af9e9a9de8fa915f P aa235ba635bf4b5ca63e16ce4aabed61: TxnStatusTablet state changed. Reason: New leader bad08fe563c74ec9bc51bf3159a4ac74. Latest consensus state: current_term: 1 leader_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } } }
I20260807 18:14:11.413527 21146 tablet_replica.cc:442] T 6208957d42d64739af9e9a9de8fa915f P aa235ba635bf4b5ca63e16ce4aabed61: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:14:11.417302 21147 tablet_replica.cc:439] T 6208957d42d64739af9e9a9de8fa915f P 067119311361489a9621056d34ddce20: TxnStatusTablet state changed. Reason: New leader bad08fe563c74ec9bc51bf3159a4ac74. Latest consensus state: current_term: 1 leader_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } } }
I20260807 18:14:11.417708 21147 tablet_replica.cc:442] T 6208957d42d64739af9e9a9de8fa915f P 067119311361489a9621056d34ddce20: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:14:11.428071 21147 tablet_replica.cc:439] T 6208957d42d64739af9e9a9de8fa915f P 067119311361489a9621056d34ddce20: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } } }
I20260807 18:14:11.428553 21147 tablet_replica.cc:442] T 6208957d42d64739af9e9a9de8fa915f P 067119311361489a9621056d34ddce20: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:14:11.429857 21147 consensus_queue.cc:1035] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:14:11.433660 21146 tablet_replica.cc:439] T 6208957d42d64739af9e9a9de8fa915f P aa235ba635bf4b5ca63e16ce4aabed61: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } } }
I20260807 18:14:11.434018 21146 tablet_replica.cc:442] T 6208957d42d64739af9e9a9de8fa915f P aa235ba635bf4b5ca63e16ce4aabed61: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:14:11.441468 21168 tablet_replica.cc:439] T 6208957d42d64739af9e9a9de8fa915f P bad08fe563c74ec9bc51bf3159a4ac74: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } } }
I20260807 18:14:11.441790 21168 tablet_replica.cc:442] T 6208957d42d64739af9e9a9de8fa915f P bad08fe563c74ec9bc51bf3159a4ac74: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:14:11.443181 21143 tablet_replica.cc:439] T 6208957d42d64739af9e9a9de8fa915f P bad08fe563c74ec9bc51bf3159a4ac74: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } } }
I20260807 18:14:11.443603 21143 tablet_replica.cc:442] T 6208957d42d64739af9e9a9de8fa915f P bad08fe563c74ec9bc51bf3159a4ac74: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:14:11.447499 21151 consensus_queue.cc:1035] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:14:11.464838 21166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:11.451096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46418 (local address 127.17.110.1:35891)
0807 18:14:11.451336 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:11.451344 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:11.455543 (+ 4199us) server_negotiation.cc:407] Connection header received
0807 18:14:11.455962 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:11.455969 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:11.456106 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:11.456272 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:14:11.457040 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.458329 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.461026 (+ 2697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.461675 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.461823 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:11.462431 (+ 608us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:14:11.463431 (+ 1000us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:14:11.463625 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:11.464363 (+ 738us) server_negotiation.cc:299] Negotiation successful
0807 18:14:11.464476 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:14:11.488166 21125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:11.478903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:52053 (local address 127.17.110.3:40729)
0807 18:14:11.479183 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:11.479193 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:11.479995 (+ 802us) server_negotiation.cc:407] Connection header received
0807 18:14:11.480312 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:11.480317 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:11.480402 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:11.480489 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:14:11.481287 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.482601 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.485055 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.485649 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.485779 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:11.487109 (+ 1330us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:14:11.487262 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:14:11.487348 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:11.487719 (+ 371us) server_negotiation.cc:299] Negotiation successful
0807 18:14:11.487816 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 18:14:11.488518 21192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:11.478842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:40729 (local address 127.17.110.1:52053)
0807 18:14:11.479717 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:11.479747 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:11.480072 (+ 325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:11.480577 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:11.480582 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:11.480598 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:14:11.481005 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:11.481020 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.482814 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.482821 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:11.484751 (+ 1930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:11.484788 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.486165 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.486175 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:11.486492 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:11.486790 (+ 298us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:14:11.487388 (+ 598us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:14:11.487404 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 18:14:11.487513 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 18:14:11.487860 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":467,"threads_started":1}
I20260807 18:14:11.544337 21204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:11.535951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:35891 (local address 127.17.110.3:54791)
0807 18:14:11.536964 (+ 1013us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:11.536981 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:11.537168 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:11.537570 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:11.537576 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:11.537590 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:14:11.538113 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:11.538127 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.539421 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.539427 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:11.541340 (+ 1913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:11.541366 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.542512 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.542520 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:11.542764 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:11.542866 (+ 102us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:14:11.543489 (+ 623us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:14:11.543503 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 18:14:11.543608 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 18:14:11.543816 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":815,"thread_start_us":656,"threads_started":1}
I20260807 18:14:11.544409 21166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:11.536012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:54791 (local address 127.17.110.1:35891)
0807 18:14:11.536228 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:11.536236 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:11.537078 (+ 842us) server_negotiation.cc:407] Connection header received
0807 18:14:11.537330 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:11.537342 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:11.537440 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:11.537542 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:14:11.538349 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.539273 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.541640 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.542267 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.542446 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:11.543110 (+ 664us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:14:11.543284 (+ 174us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:14:11.543392 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:11.543889 (+ 497us) server_negotiation.cc:299] Negotiation successful
0807 18:14:11.543997 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:14:11.593294 20407 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:14:11.593640 20407 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:14:11.598551 20407 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d176896ac4b84f47add8a59c25daceec: Sending DeleteTablet for 3 replicas of tablet f5dc56a2048e43d29600bb2df2504c67
I20260807 18:14:11.600052 20605 tablet_service.cc:1514] Processing DeleteTablet for tablet f5dc56a2048e43d29600bb2df2504c67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:14:11 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:56934
I20260807 18:14:11.600184 21003 tablet_service.cc:1514] Processing DeleteTablet for tablet f5dc56a2048e43d29600bb2df2504c67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:14:11 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:56962
I20260807 18:14:11.601483 20813 tablet_service.cc:1514] Processing DeleteTablet for tablet f5dc56a2048e43d29600bb2df2504c67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:14:11 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:59840
I20260807 18:14:11.606730 21208 tablet_replica.cc:330] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20: stopping tablet replica
I20260807 18:14:11.607813 21208 raft_consensus.cc:2229] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:14:11.608768 21208 raft_consensus.cc:2258] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:14:11.609144 20407 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42440:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:14:11.610021 20407 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:14:11.611212 21209 tablet_replica.cc:330] T f5dc56a2048e43d29600bb2df2504c67 P bad08fe563c74ec9bc51bf3159a4ac74: stopping tablet replica
I20260807 18:14:11.611337 21208 ts_tablet_manager.cc:1905] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:14:11.611843 21209 raft_consensus.cc:2229] T f5dc56a2048e43d29600bb2df2504c67 P bad08fe563c74ec9bc51bf3159a4ac74 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:14:11.612407 21209 raft_consensus.cc:2258] T f5dc56a2048e43d29600bb2df2504c67 P bad08fe563c74ec9bc51bf3159a4ac74 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:14:11.620234 20605 tablet_service.cc:1467] Processing CreateTablet for tablet d291622b5e3446f79dd868b9b371feb0 (DEFAULT_TABLE table=test-table [id=a3d8c0ecd07e47aba72dccb48140f248]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:14:11.620818 20605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d291622b5e3446f79dd868b9b371feb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:11.622418 21209 ts_tablet_manager.cc:1905] T f5dc56a2048e43d29600bb2df2504c67 P bad08fe563c74ec9bc51bf3159a4ac74: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:14:11.623232 21003 tablet_service.cc:1467] Processing CreateTablet for tablet d291622b5e3446f79dd868b9b371feb0 (DEFAULT_TABLE table=test-table [id=a3d8c0ecd07e47aba72dccb48140f248]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:14:11.623819 21003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d291622b5e3446f79dd868b9b371feb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:11.630626 20812 tablet_service.cc:1467] Processing CreateTablet for tablet d291622b5e3446f79dd868b9b371feb0 (DEFAULT_TABLE table=test-table [id=a3d8c0ecd07e47aba72dccb48140f248]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:14:11.631357 20812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d291622b5e3446f79dd868b9b371feb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:11.637305 21209 ts_tablet_manager.cc:1918] T f5dc56a2048e43d29600bb2df2504c67 P bad08fe563c74ec9bc51bf3159a4ac74: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:14:11.637594 21209 log.cc:1196] T f5dc56a2048e43d29600bb2df2504c67 P bad08fe563c74ec9bc51bf3159a4ac74: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-2/wal/wals/f5dc56a2048e43d29600bb2df2504c67
I20260807 18:14:11.638303 21209 ts_tablet_manager.cc:1939] T f5dc56a2048e43d29600bb2df2504c67 P bad08fe563c74ec9bc51bf3159a4ac74: Deleting consensus metadata
I20260807 18:14:11.640388 20387 catalog_manager.cc:4841] TS bad08fe563c74ec9bc51bf3159a4ac74 (127.17.110.3:40729): tablet f5dc56a2048e43d29600bb2df2504c67 (table test-table [id=81d0e2f74a26477280894bdbf849d739]) successfully deleted
I20260807 18:14:11.660013 21218 tablet_bootstrap.cc:492] T d291622b5e3446f79dd868b9b371feb0 P 067119311361489a9621056d34ddce20: Bootstrap starting.
I20260807 18:14:11.661238 21217 tablet_bootstrap.cc:492] T d291622b5e3446f79dd868b9b371feb0 P aa235ba635bf4b5ca63e16ce4aabed61: Bootstrap starting.
I20260807 18:14:11.663200 21217 tablet_bootstrap.cc:654] T d291622b5e3446f79dd868b9b371feb0 P aa235ba635bf4b5ca63e16ce4aabed61: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:11.665373 21214 tablet_bootstrap.cc:492] T d291622b5e3446f79dd868b9b371feb0 P bad08fe563c74ec9bc51bf3159a4ac74: Bootstrap starting.
I20260807 18:14:11.667438 21214 tablet_bootstrap.cc:654] T d291622b5e3446f79dd868b9b371feb0 P bad08fe563c74ec9bc51bf3159a4ac74: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:11.668129 21211 tablet_replica.cc:330] T f5dc56a2048e43d29600bb2df2504c67 P aa235ba635bf4b5ca63e16ce4aabed61: stopping tablet replica
I20260807 18:14:11.669421 21208 ts_tablet_manager.cc:1918] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:14:11.669668 21208 log.cc:1196] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-0/wal/wals/f5dc56a2048e43d29600bb2df2504c67
I20260807 18:14:11.670331 21208 ts_tablet_manager.cc:1939] T f5dc56a2048e43d29600bb2df2504c67 P 067119311361489a9621056d34ddce20: Deleting consensus metadata
I20260807 18:14:11.671368 21218 tablet_bootstrap.cc:654] T d291622b5e3446f79dd868b9b371feb0 P 067119311361489a9621056d34ddce20: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:11.672544 20388 catalog_manager.cc:4841] TS 067119311361489a9621056d34ddce20 (127.17.110.1:35891): tablet f5dc56a2048e43d29600bb2df2504c67 (table test-table [id=81d0e2f74a26477280894bdbf849d739]) successfully deleted
I20260807 18:14:11.672899 21211 raft_consensus.cc:2229] T f5dc56a2048e43d29600bb2df2504c67 P aa235ba635bf4b5ca63e16ce4aabed61 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:14:11.673343 21211 raft_consensus.cc:2258] T f5dc56a2048e43d29600bb2df2504c67 P aa235ba635bf4b5ca63e16ce4aabed61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:14:11.675549 21211 ts_tablet_manager.cc:1905] T f5dc56a2048e43d29600bb2df2504c67 P aa235ba635bf4b5ca63e16ce4aabed61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:14:11.680382 21217 tablet_bootstrap.cc:492] T d291622b5e3446f79dd868b9b371feb0 P aa235ba635bf4b5ca63e16ce4aabed61: No bootstrap required, opened a new log
I20260807 18:14:11.680603 21217 ts_tablet_manager.cc:1397] T d291622b5e3446f79dd868b9b371feb0 P aa235ba635bf4b5ca63e16ce4aabed61: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.000s
I20260807 18:14:11.681571 21217 raft_consensus.cc:348] T d291622b5e3446f79dd868b9b371feb0 P aa235ba635bf4b5ca63e16ce4aabed61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:11.681856 21217 raft_consensus.cc:374] T d291622b5e3446f79dd868b9b371feb0 P aa235ba635bf4b5ca63e16ce4aabed61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:11.681955 21217 raft_consensus.cc:729] T d291622b5e3446f79dd868b9b371feb0 P aa235ba635bf4b5ca63e16ce4aabed61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa235ba635bf4b5ca63e16ce4aabed61, State: Initialized, Role: FOLLOWER
I20260807 18:14:11.682235 21217 consensus_queue.cc:260] T d291622b5e3446f79dd868b9b371feb0 P aa235ba635bf4b5ca63e16ce4aabed61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:11.682907 21217 ts_tablet_manager.cc:1428] T d291622b5e3446f79dd868b9b371feb0 P aa235ba635bf4b5ca63e16ce4aabed61: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 18:14:11.685166 21214 tablet_bootstrap.cc:492] T d291622b5e3446f79dd868b9b371feb0 P bad08fe563c74ec9bc51bf3159a4ac74: No bootstrap required, opened a new log
I20260807 18:14:11.685429 21214 ts_tablet_manager.cc:1397] T d291622b5e3446f79dd868b9b371feb0 P bad08fe563c74ec9bc51bf3159a4ac74: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.000s
I20260807 18:14:11.686427 21214 raft_consensus.cc:348] T d291622b5e3446f79dd868b9b371feb0 P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:11.686800 21214 raft_consensus.cc:374] T d291622b5e3446f79dd868b9b371feb0 P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:11.686985 21214 raft_consensus.cc:729] T d291622b5e3446f79dd868b9b371feb0 P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad08fe563c74ec9bc51bf3159a4ac74, State: Initialized, Role: FOLLOWER
I20260807 18:14:11.687363 21214 consensus_queue.cc:260] T d291622b5e3446f79dd868b9b371feb0 P bad08fe563c74ec9bc51bf3159a4ac74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:11.688199 21214 ts_tablet_manager.cc:1428] T d291622b5e3446f79dd868b9b371feb0 P bad08fe563c74ec9bc51bf3159a4ac74: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 18:14:11.700101 21218 tablet_bootstrap.cc:492] T d291622b5e3446f79dd868b9b371feb0 P 067119311361489a9621056d34ddce20: No bootstrap required, opened a new log
I20260807 18:14:11.700335 21218 ts_tablet_manager.cc:1397] T d291622b5e3446f79dd868b9b371feb0 P 067119311361489a9621056d34ddce20: Time spent bootstrapping tablet: real 0.040s user 0.004s sys 0.000s
I20260807 18:14:11.701318 21218 raft_consensus.cc:348] T d291622b5e3446f79dd868b9b371feb0 P 067119311361489a9621056d34ddce20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:11.701660 21218 raft_consensus.cc:374] T d291622b5e3446f79dd868b9b371feb0 P 067119311361489a9621056d34ddce20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:11.701789 21218 raft_consensus.cc:729] T d291622b5e3446f79dd868b9b371feb0 P 067119311361489a9621056d34ddce20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 067119311361489a9621056d34ddce20, State: Initialized, Role: FOLLOWER
I20260807 18:14:11.703800 21218 consensus_queue.cc:260] T d291622b5e3446f79dd868b9b371feb0 P 067119311361489a9621056d34ddce20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:11.705407 21218 ts_tablet_manager.cc:1428] T d291622b5e3446f79dd868b9b371feb0 P 067119311361489a9621056d34ddce20: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260807 18:14:11.717911 21211 ts_tablet_manager.cc:1918] T f5dc56a2048e43d29600bb2df2504c67 P aa235ba635bf4b5ca63e16ce4aabed61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:14:11.718149 21211 log.cc:1196] T f5dc56a2048e43d29600bb2df2504c67 P aa235ba635bf4b5ca63e16ce4aabed61: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-1/wal/wals/f5dc56a2048e43d29600bb2df2504c67
I20260807 18:14:11.718693 21211 ts_tablet_manager.cc:1939] T f5dc56a2048e43d29600bb2df2504c67 P aa235ba635bf4b5ca63e16ce4aabed61: Deleting consensus metadata
I20260807 18:14:11.720747 20387 catalog_manager.cc:4841] TS aa235ba635bf4b5ca63e16ce4aabed61 (127.17.110.2:35923): tablet f5dc56a2048e43d29600bb2df2504c67 (table test-table [id=81d0e2f74a26477280894bdbf849d739]) successfully deleted
I20260807 18:14:11.847257 21146 raft_consensus.cc:482] T d291622b5e3446f79dd868b9b371feb0 P aa235ba635bf4b5ca63e16ce4aabed61 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:14:11.847430 21184 raft_consensus.cc:482] T d291622b5e3446f79dd868b9b371feb0 P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:14:11.847513 21146 raft_consensus.cc:504] T d291622b5e3446f79dd868b9b371feb0 P aa235ba635bf4b5ca63e16ce4aabed61 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:11.847647 21184 raft_consensus.cc:504] T d291622b5e3446f79dd868b9b371feb0 P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:11.848515 21184 leader_election.cc:288] T d291622b5e3446f79dd868b9b371feb0 P bad08fe563c74ec9bc51bf3159a4ac74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 067119311361489a9621056d34ddce20 (127.17.110.1:35891), aa235ba635bf4b5ca63e16ce4aabed61 (127.17.110.2:35923)
I20260807 18:14:11.849401 21146 leader_election.cc:288] T d291622b5e3446f79dd868b9b371feb0 P aa235ba635bf4b5ca63e16ce4aabed61 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 067119311361489a9621056d34ddce20 (127.17.110.1:35891), bad08fe563c74ec9bc51bf3159a4ac74 (127.17.110.3:40729)
I20260807 18:14:11.849429 20638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d291622b5e3446f79dd868b9b371feb0" candidate_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "067119311361489a9621056d34ddce20" is_pre_election: true
I20260807 18:14:11.849740 20638 raft_consensus.cc:2454] T d291622b5e3446f79dd868b9b371feb0 P 067119311361489a9621056d34ddce20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bad08fe563c74ec9bc51bf3159a4ac74 in term 0.
I20260807 18:14:11.850328 20940 leader_election.cc:302] T d291622b5e3446f79dd868b9b371feb0 P bad08fe563c74ec9bc51bf3159a4ac74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 067119311361489a9621056d34ddce20, bad08fe563c74ec9bc51bf3159a4ac74; no voters:
I20260807 18:14:11.851137 21184 raft_consensus.cc:2781] T d291622b5e3446f79dd868b9b371feb0 P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:14:11.851296 21184 raft_consensus.cc:482] T d291622b5e3446f79dd868b9b371feb0 P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:14:11.851419 21184 raft_consensus.cc:3037] T d291622b5e3446f79dd868b9b371feb0 P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:11.854974 20840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d291622b5e3446f79dd868b9b371feb0" candidate_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" is_pre_election: true
I20260807 18:14:11.855288 20840 raft_consensus.cc:2454] T d291622b5e3446f79dd868b9b371feb0 P aa235ba635bf4b5ca63e16ce4aabed61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bad08fe563c74ec9bc51bf3159a4ac74 in term 0.
I20260807 18:14:11.856451 21184 raft_consensus.cc:504] T d291622b5e3446f79dd868b9b371feb0 P bad08fe563c74ec9bc51bf3159a4ac74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:11.857969 20840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d291622b5e3446f79dd868b9b371feb0" candidate_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa235ba635bf4b5ca63e16ce4aabed61"
I20260807 18:14:11.858038 20638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d291622b5e3446f79dd868b9b371feb0" candidate_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "067119311361489a9621056d34ddce20"
I20260807 18:14:11.858223 20840 raft_consensus.cc:3037] T d291622b5e3446f79dd868b9b371feb0 P aa235ba635bf4b5ca63e16ce4aabed61 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:11.858321 20638 raft_consensus.cc:3037] T d291622b5e3446f79dd868b9b371feb0 P 067119311361489a9621056d34ddce20 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:11.861649 20840 raft_consensus.cc:2454] T d291622b5e3446f79dd868b9b371feb0 P aa235ba635bf4b5ca63e16ce4aabed61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bad08fe563c74ec9bc51bf3159a4ac74 in term 1.
I20260807 18:14:11.862293 20939 leader_election.cc:302] T d291622b5e3446f79dd868b9b371feb0 P bad08fe563c74ec9bc51bf3159a4ac74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa235ba635bf4b5ca63e16ce4aabed61, bad08fe563c74ec9bc51bf3159a4ac74; no voters:
I20260807 18:14:11.862728 21195 raft_consensus.cc:2781] T d291622b5e3446f79dd868b9b371feb0 P bad08fe563c74ec9bc51bf3159a4ac74 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:14:11.862907 21195 raft_consensus.cc:686] T d291622b5e3446f79dd868b9b371feb0 P bad08fe563c74ec9bc51bf3159a4ac74 [term 1 LEADER]: Becoming Leader. State: Replica: bad08fe563c74ec9bc51bf3159a4ac74, State: Running, Role: LEADER
I20260807 18:14:11.863251 21195 consensus_queue.cc:237] T d291622b5e3446f79dd868b9b371feb0 P bad08fe563c74ec9bc51bf3159a4ac74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:11.866055 20638 raft_consensus.cc:2454] T d291622b5e3446f79dd868b9b371feb0 P 067119311361489a9621056d34ddce20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bad08fe563c74ec9bc51bf3159a4ac74 in term 1.
I20260807 18:14:11.866503 21184 leader_election.cc:288] T d291622b5e3446f79dd868b9b371feb0 P bad08fe563c74ec9bc51bf3159a4ac74 [CANDIDATE]: Term 1 election: Requested vote from peers 067119311361489a9621056d34ddce20 (127.17.110.1:35891), aa235ba635bf4b5ca63e16ce4aabed61 (127.17.110.2:35923)
I20260807 18:14:11.875622 20407 catalog_manager.cc:5495] T d291622b5e3446f79dd868b9b371feb0 P bad08fe563c74ec9bc51bf3159a4ac74 reported cstate change: term changed from 0 to 1, leader changed from <none> to bad08fe563c74ec9bc51bf3159a4ac74 (127.17.110.3). New cstate: current_term: 1 leader_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } health_report { overall_health: HEALTHY } } }
I20260807 18:14:11.884277 21166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:11.850760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:59827 (local address 127.17.110.1:35891)
0807 18:14:11.850990 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:11.850996 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:11.875043 (+ 24047us) server_negotiation.cc:407] Connection header received
0807 18:14:11.875514 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:11.875520 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:11.875618 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:11.875695 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:14:11.877016 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.878291 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.881072 (+ 2781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.882594 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.882819 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:11.882920 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:11.883684 (+ 764us) server_negotiation.cc:299] Negotiation successful
0807 18:14:11.883870 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:14:11.885251 21225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:11.849908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:35891 (local address 127.17.110.2:59827)
0807 18:14:11.874817 (+ 24909us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:11.874846 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:11.875292 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:11.876481 (+ 1189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:11.876487 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:11.876521 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:14:11.876780 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:11.876792 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.878529 (+ 1737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.878535 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:11.880833 (+ 2298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:11.880854 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.883159 (+ 2305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.883166 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:11.883447 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:11.883491 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 18:14:11.884701 (+ 1210us) client_negotiation.cc:240] Negotiation successful
0807 18:14:11.884877 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24686,"threads_started":1}
I20260807 18:14:11.891985 20638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d291622b5e3446f79dd868b9b371feb0" candidate_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "067119311361489a9621056d34ddce20" is_pre_election: true
I20260807 18:14:11.892598 20638 raft_consensus.cc:2379] T d291622b5e3446f79dd868b9b371feb0 P 067119311361489a9621056d34ddce20 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aa235ba635bf4b5ca63e16ce4aabed61 in current term 1: Already voted for candidate bad08fe563c74ec9bc51bf3159a4ac74 in this term.
I20260807 18:14:11.902868 21125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:11.852507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:35213 (local address 127.17.110.3:40729)
0807 18:14:11.852713 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:11.852720 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:11.885651 (+ 32931us) server_negotiation.cc:407] Connection header received
0807 18:14:11.885946 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:11.885953 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:11.886037 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:11.886102 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:14:11.886753 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.888180 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.899276 (+ 11096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.901307 (+ 2031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.901486 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:11.901566 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:11.902302 (+ 736us) server_negotiation.cc:299] Negotiation successful
0807 18:14:11.902509 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 18:14:11.911119 21125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:11.900631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36116 (local address 127.17.110.3:40729)
0807 18:14:11.903387 (+ 2756us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:11.903395 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:11.903416 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:14:11.903487 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:11.903492 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:11.903587 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:11.904269 (+ 682us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:14:11.904439 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.905589 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.907900 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.908639 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.909493 (+ 854us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:11.909641 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:14:11.909812 (+ 171us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:14:11.910558 (+ 746us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:11.910614 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 18:14:11.910768 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2631,"threads_started":1}
I20260807 18:14:11.912176 21225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:11.850454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:40729 (local address 127.17.110.2:35213)
0807 18:14:11.885534 (+ 35080us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:11.885556 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:11.885777 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:11.886325 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:11.886331 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:11.886359 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:14:11.886548 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:11.886561 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.896791 (+ 10230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.896797 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:11.898943 (+ 2146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:11.898967 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.901794 (+ 2827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.901803 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:11.902058 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:11.902101 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 18:14:11.911596 (+ 9495us) client_negotiation.cc:240] Negotiation successful
0807 18:14:11.911818 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34921,"thread_start_us":39510,"threads_started":1}
I20260807 18:14:11.915069 21036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d291622b5e3446f79dd868b9b371feb0" candidate_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" is_pre_election: true
I20260807 18:14:11.915930 20743 leader_election.cc:302] T d291622b5e3446f79dd868b9b371feb0 P aa235ba635bf4b5ca63e16ce4aabed61 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa235ba635bf4b5ca63e16ce4aabed61; no voters: 067119311361489a9621056d34ddce20, bad08fe563c74ec9bc51bf3159a4ac74
I20260807 18:14:11.916428 21146 raft_consensus.cc:2726] T d291622b5e3446f79dd868b9b371feb0 P aa235ba635bf4b5ca63e16ce4aabed61 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 18:14:11.931769 21125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:11.924407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:44675 (local address 127.17.110.3:40729)
0807 18:14:11.924623 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:11.924629 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:11.924649 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:14:11.924710 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:11.924714 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:11.924805 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:11.924903 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:14:11.925382 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.926574 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.929163 (+ 2589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.929921 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.930088 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:11.930740 (+ 652us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:14:11.930956 (+ 216us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:14:11.931065 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:11.931289 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 18:14:11.931381 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:14:11.931993 21204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:11.924093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:40729 (local address 127.17.110.3:44675)
0807 18:14:11.924346 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:11.924370 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:11.924530 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:11.924971 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:11.924977 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:11.924994 (+ 17us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:14:11.925218 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:11.925230 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.926779 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.926787 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:11.928886 (+ 2099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:11.928907 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:11.930185 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:11.930194 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:11.930442 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:11.930503 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:14:11.931099 (+ 596us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:14:11.931116 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 18:14:11.931247 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 18:14:11.931472 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 18:14:11.943275 20840 raft_consensus.cc:1261] T d291622b5e3446f79dd868b9b371feb0 P aa235ba635bf4b5ca63e16ce4aabed61 [term 1 FOLLOWER]: Refusing update from remote peer bad08fe563c74ec9bc51bf3159a4ac74: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:14:11.943496 20638 raft_consensus.cc:1261] T d291622b5e3446f79dd868b9b371feb0 P 067119311361489a9621056d34ddce20 [term 1 FOLLOWER]: Refusing update from remote peer bad08fe563c74ec9bc51bf3159a4ac74: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:14:11.944087 21143 consensus_queue.cc:1035] T d291622b5e3446f79dd868b9b371feb0 P bad08fe563c74ec9bc51bf3159a4ac74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:14:11.944430 21184 consensus_queue.cc:1035] T d291622b5e3446f79dd868b9b371feb0 P bad08fe563c74ec9bc51bf3159a4ac74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:14:11.953892 21173 mvcc.cc:205] Tried to move back new op lower bound from 7315973947153051648 to 7315973946835025920. Current Snapshot: MvccSnapshot[applied={T|T < 7315973947153051648}]
W20260807 18:14:11.976708 21173 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 18:14:13.601670 20408 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:14:13.601979 20408 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:14:13.607059 20408 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d176896ac4b84f47add8a59c25daceec: Sending DeleteTablet for 3 replicas of tablet d291622b5e3446f79dd868b9b371feb0
I20260807 18:14:13.608107 20605 tablet_service.cc:1514] Processing DeleteTablet for tablet d291622b5e3446f79dd868b9b371feb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:14:13 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:56934
I20260807 18:14:13.608161 20813 tablet_service.cc:1514] Processing DeleteTablet for tablet d291622b5e3446f79dd868b9b371feb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:14:13 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:59840
I20260807 18:14:13.608290 21003 tablet_service.cc:1514] Processing DeleteTablet for tablet d291622b5e3446f79dd868b9b371feb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:14:13 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:56962
I20260807 18:14:13.620867 21267 tablet_replica.cc:330] T d291622b5e3446f79dd868b9b371feb0 P 067119311361489a9621056d34ddce20: stopping tablet replica
I20260807 18:14:13.621471 21267 raft_consensus.cc:2229] T d291622b5e3446f79dd868b9b371feb0 P 067119311361489a9621056d34ddce20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:14:13.621963 21267 raft_consensus.cc:2258] T d291622b5e3446f79dd868b9b371feb0 P 067119311361489a9621056d34ddce20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:14:13.623026 21268 tablet_replica.cc:330] T d291622b5e3446f79dd868b9b371feb0 P aa235ba635bf4b5ca63e16ce4aabed61: stopping tablet replica
I20260807 18:14:13.624432 21267 ts_tablet_manager.cc:1905] T d291622b5e3446f79dd868b9b371feb0 P 067119311361489a9621056d34ddce20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:14:13.629983 21268 raft_consensus.cc:2229] T d291622b5e3446f79dd868b9b371feb0 P aa235ba635bf4b5ca63e16ce4aabed61 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:14:13.630110 21269 tablet_replica.cc:330] T d291622b5e3446f79dd868b9b371feb0 P bad08fe563c74ec9bc51bf3159a4ac74: stopping tablet replica
I20260807 18:14:13.630327 21269 raft_consensus.cc:2229] T d291622b5e3446f79dd868b9b371feb0 P bad08fe563c74ec9bc51bf3159a4ac74 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:14:13.630376 21268 raft_consensus.cc:2258] T d291622b5e3446f79dd868b9b371feb0 P aa235ba635bf4b5ca63e16ce4aabed61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:14:13.631415 21269 raft_consensus.cc:2258] T d291622b5e3446f79dd868b9b371feb0 P bad08fe563c74ec9bc51bf3159a4ac74 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:14:13.632865 21269 ts_tablet_manager.cc:1905] T d291622b5e3446f79dd868b9b371feb0 P bad08fe563c74ec9bc51bf3159a4ac74: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:14:13.634506 21268 ts_tablet_manager.cc:1905] T d291622b5e3446f79dd868b9b371feb0 P aa235ba635bf4b5ca63e16ce4aabed61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:14:13.641360 21269 ts_tablet_manager.cc:1918] T d291622b5e3446f79dd868b9b371feb0 P bad08fe563c74ec9bc51bf3159a4ac74: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:14:13.641664 21269 log.cc:1196] T d291622b5e3446f79dd868b9b371feb0 P bad08fe563c74ec9bc51bf3159a4ac74: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-2/wal/wals/d291622b5e3446f79dd868b9b371feb0
I20260807 18:14:13.641858 21268 ts_tablet_manager.cc:1918] T d291622b5e3446f79dd868b9b371feb0 P aa235ba635bf4b5ca63e16ce4aabed61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:14:13.642305 21269 ts_tablet_manager.cc:1939] T d291622b5e3446f79dd868b9b371feb0 P bad08fe563c74ec9bc51bf3159a4ac74: Deleting consensus metadata
I20260807 18:14:13.644258 20387 catalog_manager.cc:4841] TS bad08fe563c74ec9bc51bf3159a4ac74 (127.17.110.3:40729): tablet d291622b5e3446f79dd868b9b371feb0 (table test-table [id=a3d8c0ecd07e47aba72dccb48140f248]) successfully deleted
I20260807 18:14:13.644742 21268 log.cc:1196] T d291622b5e3446f79dd868b9b371feb0 P aa235ba635bf4b5ca63e16ce4aabed61: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-1/wal/wals/d291622b5e3446f79dd868b9b371feb0
I20260807 18:14:13.645443 21268 ts_tablet_manager.cc:1939] T d291622b5e3446f79dd868b9b371feb0 P aa235ba635bf4b5ca63e16ce4aabed61: Deleting consensus metadata
I20260807 18:14:13.646055 21267 ts_tablet_manager.cc:1918] T d291622b5e3446f79dd868b9b371feb0 P 067119311361489a9621056d34ddce20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:14:13.646291 21267 log.cc:1196] T d291622b5e3446f79dd868b9b371feb0 P 067119311361489a9621056d34ddce20: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-0/wal/wals/d291622b5e3446f79dd868b9b371feb0
I20260807 18:14:13.646874 21267 ts_tablet_manager.cc:1939] T d291622b5e3446f79dd868b9b371feb0 P 067119311361489a9621056d34ddce20: Deleting consensus metadata
I20260807 18:14:13.646870 20387 catalog_manager.cc:4841] TS aa235ba635bf4b5ca63e16ce4aabed61 (127.17.110.2:35923): tablet d291622b5e3446f79dd868b9b371feb0 (table test-table [id=a3d8c0ecd07e47aba72dccb48140f248]) successfully deleted
I20260807 18:14:13.652304 20388 catalog_manager.cc:4841] TS 067119311361489a9621056d34ddce20 (127.17.110.1:35891): tablet d291622b5e3446f79dd868b9b371feb0 (table test-table [id=a3d8c0ecd07e47aba72dccb48140f248]) successfully deleted
Aug 07 18:14:13 dist-test-slave-39l8 krb5kdc[20345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126453, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 18:14:13 dist-test-slave-39l8 krb5kdc[20345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126453, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:13.703634 21278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:13.685029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60460 (local address 127.17.110.62:44799)
0807 18:14:13.686106 (+ 1077us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:13.686110 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:13.686132 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:14:13.686190 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:13.686195 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:13.686256 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:13.686355 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:13.688483 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:13.689461 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:13.691129 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:13.691642 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:13.691814 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:13.696727 (+ 4913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:13.696734 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:13.696738 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:13.696789 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:13.700348 (+ 3559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:13.701228 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:13.701234 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:13.701236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:13.701399 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:13.701885 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:13.701890 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:13.701892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:13.702215 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:13.702498 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:13.702907 (+ 409us) server_negotiation.cc:299] Negotiation successful
0807 18:14:13.703179 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":954,"thread_start_us":796,"threads_started":1}
I20260807 18:14:13.708226 20408 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60460:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:14:13.708979 20408 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:14:13.717142 20605 tablet_service.cc:1467] Processing CreateTablet for tablet 44c5af415846459a92649e0a60513356 (DEFAULT_TABLE table=test-table [id=82bce5e4ad50411e8e2bbbc1dac0c7d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:14:13.717262 20813 tablet_service.cc:1467] Processing CreateTablet for tablet 44c5af415846459a92649e0a60513356 (DEFAULT_TABLE table=test-table [id=82bce5e4ad50411e8e2bbbc1dac0c7d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:14:13.717727 20605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44c5af415846459a92649e0a60513356. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:13.717778 20813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44c5af415846459a92649e0a60513356. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:13.717665 21003 tablet_service.cc:1467] Processing CreateTablet for tablet 44c5af415846459a92649e0a60513356 (DEFAULT_TABLE table=test-table [id=82bce5e4ad50411e8e2bbbc1dac0c7d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:14:13.718119 21003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44c5af415846459a92649e0a60513356. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:13.725160 21279 tablet_bootstrap.cc:492] T 44c5af415846459a92649e0a60513356 P 067119311361489a9621056d34ddce20: Bootstrap starting.
I20260807 18:14:13.726331 21280 tablet_bootstrap.cc:492] T 44c5af415846459a92649e0a60513356 P bad08fe563c74ec9bc51bf3159a4ac74: Bootstrap starting.
I20260807 18:14:13.727324 21281 tablet_bootstrap.cc:492] T 44c5af415846459a92649e0a60513356 P aa235ba635bf4b5ca63e16ce4aabed61: Bootstrap starting.
I20260807 18:14:13.728549 21280 tablet_bootstrap.cc:654] T 44c5af415846459a92649e0a60513356 P bad08fe563c74ec9bc51bf3159a4ac74: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:13.728535 21279 tablet_bootstrap.cc:654] T 44c5af415846459a92649e0a60513356 P 067119311361489a9621056d34ddce20: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:13.729429 21281 tablet_bootstrap.cc:654] T 44c5af415846459a92649e0a60513356 P aa235ba635bf4b5ca63e16ce4aabed61: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:13.731483 21279 tablet_bootstrap.cc:492] T 44c5af415846459a92649e0a60513356 P 067119311361489a9621056d34ddce20: No bootstrap required, opened a new log
I20260807 18:14:13.731612 21280 tablet_bootstrap.cc:492] T 44c5af415846459a92649e0a60513356 P bad08fe563c74ec9bc51bf3159a4ac74: No bootstrap required, opened a new log
I20260807 18:14:13.731706 21279 ts_tablet_manager.cc:1397] T 44c5af415846459a92649e0a60513356 P 067119311361489a9621056d34ddce20: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 18:14:13.731839 21280 ts_tablet_manager.cc:1397] T 44c5af415846459a92649e0a60513356 P bad08fe563c74ec9bc51bf3159a4ac74: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 18:14:13.732110 21281 tablet_bootstrap.cc:492] T 44c5af415846459a92649e0a60513356 P aa235ba635bf4b5ca63e16ce4aabed61: No bootstrap required, opened a new log
I20260807 18:14:13.732285 21281 ts_tablet_manager.cc:1397] T 44c5af415846459a92649e0a60513356 P aa235ba635bf4b5ca63e16ce4aabed61: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20260807 18:14:13.732743 21280 raft_consensus.cc:348] T 44c5af415846459a92649e0a60513356 P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:13.732687 21279 raft_consensus.cc:348] T 44c5af415846459a92649e0a60513356 P 067119311361489a9621056d34ddce20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:13.732999 21279 raft_consensus.cc:374] T 44c5af415846459a92649e0a60513356 P 067119311361489a9621056d34ddce20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:13.732999 21280 raft_consensus.cc:374] T 44c5af415846459a92649e0a60513356 P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:13.733108 21279 raft_consensus.cc:729] T 44c5af415846459a92649e0a60513356 P 067119311361489a9621056d34ddce20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 067119311361489a9621056d34ddce20, State: Initialized, Role: FOLLOWER
I20260807 18:14:13.733117 21280 raft_consensus.cc:729] T 44c5af415846459a92649e0a60513356 P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad08fe563c74ec9bc51bf3159a4ac74, State: Initialized, Role: FOLLOWER
I20260807 18:14:13.733170 21281 raft_consensus.cc:348] T 44c5af415846459a92649e0a60513356 P aa235ba635bf4b5ca63e16ce4aabed61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:13.733520 21281 raft_consensus.cc:374] T 44c5af415846459a92649e0a60513356 P aa235ba635bf4b5ca63e16ce4aabed61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:13.733433 21280 consensus_queue.cc:260] T 44c5af415846459a92649e0a60513356 P bad08fe563c74ec9bc51bf3159a4ac74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:13.733613 21281 raft_consensus.cc:729] T 44c5af415846459a92649e0a60513356 P aa235ba635bf4b5ca63e16ce4aabed61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa235ba635bf4b5ca63e16ce4aabed61, State: Initialized, Role: FOLLOWER
I20260807 18:14:13.733405 21279 consensus_queue.cc:260] T 44c5af415846459a92649e0a60513356 P 067119311361489a9621056d34ddce20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:13.733904 21281 consensus_queue.cc:260] T 44c5af415846459a92649e0a60513356 P aa235ba635bf4b5ca63e16ce4aabed61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:13.734251 21280 ts_tablet_manager.cc:1428] T 44c5af415846459a92649e0a60513356 P bad08fe563c74ec9bc51bf3159a4ac74: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 18:14:13.735172 21279 ts_tablet_manager.cc:1428] T 44c5af415846459a92649e0a60513356 P 067119311361489a9621056d34ddce20: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 18:14:13.737099 21281 ts_tablet_manager.cc:1428] T 44c5af415846459a92649e0a60513356 P aa235ba635bf4b5ca63e16ce4aabed61: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20260807 18:14:13.969993 21289 raft_consensus.cc:482] T 44c5af415846459a92649e0a60513356 P aa235ba635bf4b5ca63e16ce4aabed61 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:14:13.970350 21289 raft_consensus.cc:504] T 44c5af415846459a92649e0a60513356 P aa235ba635bf4b5ca63e16ce4aabed61 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:13.971323 21289 leader_election.cc:288] T 44c5af415846459a92649e0a60513356 P aa235ba635bf4b5ca63e16ce4aabed61 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 067119311361489a9621056d34ddce20 (127.17.110.1:35891), bad08fe563c74ec9bc51bf3159a4ac74 (127.17.110.3:40729)
I20260807 18:14:13.972064 21036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44c5af415846459a92649e0a60513356" candidate_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" is_pre_election: true
I20260807 18:14:13.972121 20636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44c5af415846459a92649e0a60513356" candidate_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "067119311361489a9621056d34ddce20" is_pre_election: true
I20260807 18:14:13.972352 21036 raft_consensus.cc:2454] T 44c5af415846459a92649e0a60513356 P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa235ba635bf4b5ca63e16ce4aabed61 in term 0.
I20260807 18:14:13.972587 20636 raft_consensus.cc:2454] T 44c5af415846459a92649e0a60513356 P 067119311361489a9621056d34ddce20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa235ba635bf4b5ca63e16ce4aabed61 in term 0.
I20260807 18:14:13.973171 20743 leader_election.cc:302] T 44c5af415846459a92649e0a60513356 P aa235ba635bf4b5ca63e16ce4aabed61 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa235ba635bf4b5ca63e16ce4aabed61, bad08fe563c74ec9bc51bf3159a4ac74; no voters:
I20260807 18:14:13.973651 21289 raft_consensus.cc:2781] T 44c5af415846459a92649e0a60513356 P aa235ba635bf4b5ca63e16ce4aabed61 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:14:13.973824 21289 raft_consensus.cc:482] T 44c5af415846459a92649e0a60513356 P aa235ba635bf4b5ca63e16ce4aabed61 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:14:13.973973 21289 raft_consensus.cc:3037] T 44c5af415846459a92649e0a60513356 P aa235ba635bf4b5ca63e16ce4aabed61 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:13.976982 21289 raft_consensus.cc:504] T 44c5af415846459a92649e0a60513356 P aa235ba635bf4b5ca63e16ce4aabed61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:13.977843 21289 leader_election.cc:288] T 44c5af415846459a92649e0a60513356 P aa235ba635bf4b5ca63e16ce4aabed61 [CANDIDATE]: Term 1 election: Requested vote from peers 067119311361489a9621056d34ddce20 (127.17.110.1:35891), bad08fe563c74ec9bc51bf3159a4ac74 (127.17.110.3:40729)
I20260807 18:14:13.978578 21036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44c5af415846459a92649e0a60513356" candidate_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad08fe563c74ec9bc51bf3159a4ac74"
I20260807 18:14:13.978600 20636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44c5af415846459a92649e0a60513356" candidate_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "067119311361489a9621056d34ddce20"
I20260807 18:14:13.978875 20636 raft_consensus.cc:3037] T 44c5af415846459a92649e0a60513356 P 067119311361489a9621056d34ddce20 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:13.978875 21036 raft_consensus.cc:3037] T 44c5af415846459a92649e0a60513356 P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:13.982493 20636 raft_consensus.cc:2454] T 44c5af415846459a92649e0a60513356 P 067119311361489a9621056d34ddce20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa235ba635bf4b5ca63e16ce4aabed61 in term 1.
I20260807 18:14:13.982493 21036 raft_consensus.cc:2454] T 44c5af415846459a92649e0a60513356 P bad08fe563c74ec9bc51bf3159a4ac74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa235ba635bf4b5ca63e16ce4aabed61 in term 1.
I20260807 18:14:13.983289 20743 leader_election.cc:302] T 44c5af415846459a92649e0a60513356 P aa235ba635bf4b5ca63e16ce4aabed61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 067119311361489a9621056d34ddce20, aa235ba635bf4b5ca63e16ce4aabed61, bad08fe563c74ec9bc51bf3159a4ac74; no voters:
I20260807 18:14:13.983799 21289 raft_consensus.cc:2781] T 44c5af415846459a92649e0a60513356 P aa235ba635bf4b5ca63e16ce4aabed61 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:14:13.984421 21289 raft_consensus.cc:686] T 44c5af415846459a92649e0a60513356 P aa235ba635bf4b5ca63e16ce4aabed61 [term 1 LEADER]: Becoming Leader. State: Replica: aa235ba635bf4b5ca63e16ce4aabed61, State: Running, Role: LEADER
I20260807 18:14:13.984984 21289 consensus_queue.cc:237] T 44c5af415846459a92649e0a60513356 P aa235ba635bf4b5ca63e16ce4aabed61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:13.989454 20408 catalog_manager.cc:5495] T 44c5af415846459a92649e0a60513356 P aa235ba635bf4b5ca63e16ce4aabed61 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa235ba635bf4b5ca63e16ce4aabed61 (127.17.110.2). New cstate: current_term: 1 leader_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } health_report { overall_health: UNKNOWN } } }
I20260807 18:14:14.043896 21295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:14.037623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56876 (local address 127.17.110.2:35923)
0807 18:14:14.038313 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:14.038317 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:14.038338 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:14:14.038427 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:14.038431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:14.038511 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:14.038670 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:14:14.039201 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:14.040008 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:14.041363 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:14.041808 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:14.041917 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:14.042400 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:14:14.043171 (+ 771us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:14:14.043297 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:14.043435 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 18:14:14.043510 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":385,"threads_started":1}
I20260807 18:14:14.063212 21299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:14.056241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:55013 (local address 127.17.110.3:40729)
0807 18:14:14.056948 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:14.056957 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:14.057081 (+ 124us) server_negotiation.cc:407] Connection header received
0807 18:14:14.057374 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:14.057378 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:14.057456 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:14.057577 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:14:14.058267 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:14.059207 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:14.060827 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:14.061462 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:14.061646 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:14.062197 (+ 551us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:14:14.062381 (+ 184us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:14:14.062492 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:14.062674 (+ 182us) server_negotiation.cc:299] Negotiation successful
0807 18:14:14.062797 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":352,"threads_started":1}
I20260807 18:14:14.063259 21298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:14.055954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:40729 (local address 127.17.110.2:55013)
0807 18:14:14.056853 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:14.056970 (+ 117us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:14.057200 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:14.057600 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:14.057605 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:14.057620 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:14:14.058014 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:14.058026 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:14.059360 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:14.059366 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:14.060649 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:14.060664 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:14.061688 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:14.061697 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:14.061870 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:14.061943 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:14:14.062498 (+ 555us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:14:14.062514 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 18:14:14.062629 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 18:14:14.062825 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":430,"threads_started":1}
I20260807 18:14:14.073439 20636 raft_consensus.cc:1261] T 44c5af415846459a92649e0a60513356 P 067119311361489a9621056d34ddce20 [term 1 FOLLOWER]: Refusing update from remote peer aa235ba635bf4b5ca63e16ce4aabed61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:14:14.073546 21036 raft_consensus.cc:1261] T 44c5af415846459a92649e0a60513356 P bad08fe563c74ec9bc51bf3159a4ac74 [term 1 FOLLOWER]: Refusing update from remote peer aa235ba635bf4b5ca63e16ce4aabed61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:14:14.074175 21292 consensus_queue.cc:1035] T 44c5af415846459a92649e0a60513356 P aa235ba635bf4b5ca63e16ce4aabed61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:14:14.074642 21289 consensus_queue.cc:1035] T 44c5af415846459a92649e0a60513356 P aa235ba635bf4b5ca63e16ce4aabed61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:14:14.083173 21242 mvcc.cc:205] Tried to move back new op lower bound from 7315973955877978112 to 7315973955527528448. Current Snapshot: MvccSnapshot[applied={T|T < 7315973955877978112}]
I20260807 18:14:14.112847 21295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:14.106080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:43199 (local address 127.17.110.2:35923)
0807 18:14:14.106330 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:14.106338 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:14.106766 (+ 428us) server_negotiation.cc:407] Connection header received
0807 18:14:14.106863 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:14.106868 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:14.107012 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:14.107138 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:14:14.107594 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:14.108553 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:14.110512 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:14.111302 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:14.111473 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:14.111982 (+ 509us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:14:14.112136 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:14:14.112278 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:14.112518 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 18:14:14.112598 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260807 18:14:14.112892 21305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:14.105648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:35923 (local address 127.17.110.3:43199)
0807 18:14:14.106414 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:14.106439 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:14.106698 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:14.107164 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:14.107170 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:14.107192 (+ 22us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:14:14.107453 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:14.107462 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:14.108801 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:14.108807 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:14.110333 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:14.110347 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:14.111539 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:14.111545 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:14.111734 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:14.111781 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:14:14.112314 (+ 533us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:14:14.112336 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 18:14:14.112444 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 18:14:14.112572 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":591,"threads_started":1}
I20260807 18:14:14.153187 20408 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:14:14.153626 20408 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:14:14.157752 20408 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d176896ac4b84f47add8a59c25daceec: Sending DeleteTablet for 3 replicas of tablet 44c5af415846459a92649e0a60513356
I20260807 18:14:14.158943 20605 tablet_service.cc:1514] Processing DeleteTablet for tablet 44c5af415846459a92649e0a60513356 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:14:14 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:56934
I20260807 18:14:14.158939 20813 tablet_service.cc:1514] Processing DeleteTablet for tablet 44c5af415846459a92649e0a60513356 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:14:14 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:59840
I20260807 18:14:14.159973 21003 tablet_service.cc:1514] Processing DeleteTablet for tablet 44c5af415846459a92649e0a60513356 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:14:14 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:56962
I20260807 18:14:14.162482 20408 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60460:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:14:14.163612 20408 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:14:14.168555 21312 tablet_replica.cc:330] T 44c5af415846459a92649e0a60513356 P bad08fe563c74ec9bc51bf3159a4ac74: stopping tablet replica
I20260807 18:14:14.169009 21312 raft_consensus.cc:2229] T 44c5af415846459a92649e0a60513356 P bad08fe563c74ec9bc51bf3159a4ac74 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:14:14.169443 21312 raft_consensus.cc:2258] T 44c5af415846459a92649e0a60513356 P bad08fe563c74ec9bc51bf3159a4ac74 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:14:14.171027 21312 ts_tablet_manager.cc:1905] T 44c5af415846459a92649e0a60513356 P bad08fe563c74ec9bc51bf3159a4ac74: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:14:14.171442 21309 tablet_replica.cc:330] T 44c5af415846459a92649e0a60513356 P 067119311361489a9621056d34ddce20: stopping tablet replica
I20260807 18:14:14.171700 21309 raft_consensus.cc:2229] T 44c5af415846459a92649e0a60513356 P 067119311361489a9621056d34ddce20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:14:14.171967 21309 raft_consensus.cc:2258] T 44c5af415846459a92649e0a60513356 P 067119311361489a9621056d34ddce20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:14:14.173463 21309 ts_tablet_manager.cc:1905] T 44c5af415846459a92649e0a60513356 P 067119311361489a9621056d34ddce20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:14:14.175365 21003 tablet_service.cc:1467] Processing CreateTablet for tablet c5972b7469d745d69312e6788a9c2dc0 (DEFAULT_TABLE table=test-table [id=1a763edff4bf4de5a8b0c2234fe116f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:14:14.175614 20812 tablet_service.cc:1467] Processing CreateTablet for tablet c5972b7469d745d69312e6788a9c2dc0 (DEFAULT_TABLE table=test-table [id=1a763edff4bf4de5a8b0c2234fe116f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:14:14.175861 21003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5972b7469d745d69312e6788a9c2dc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:14.175810 20605 tablet_service.cc:1467] Processing CreateTablet for tablet c5972b7469d745d69312e6788a9c2dc0 (DEFAULT_TABLE table=test-table [id=1a763edff4bf4de5a8b0c2234fe116f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:14:14.176107 20812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5972b7469d745d69312e6788a9c2dc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:14.176271 20605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5972b7469d745d69312e6788a9c2dc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:14.181974 21309 ts_tablet_manager.cc:1918] T 44c5af415846459a92649e0a60513356 P 067119311361489a9621056d34ddce20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:14:14.182164 21309 log.cc:1196] T 44c5af415846459a92649e0a60513356 P 067119311361489a9621056d34ddce20: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-0/wal/wals/44c5af415846459a92649e0a60513356
I20260807 18:14:14.182802 21309 ts_tablet_manager.cc:1939] T 44c5af415846459a92649e0a60513356 P 067119311361489a9621056d34ddce20: Deleting consensus metadata
I20260807 18:14:14.183449 21280 tablet_bootstrap.cc:492] T c5972b7469d745d69312e6788a9c2dc0 P bad08fe563c74ec9bc51bf3159a4ac74: Bootstrap starting.
I20260807 18:14:14.183897 21279 tablet_bootstrap.cc:492] T c5972b7469d745d69312e6788a9c2dc0 P 067119311361489a9621056d34ddce20: Bootstrap starting.
I20260807 18:14:14.184140 20388 catalog_manager.cc:4841] TS 067119311361489a9621056d34ddce20 (127.17.110.1:35891): tablet 44c5af415846459a92649e0a60513356 (table test-table [id=82bce5e4ad50411e8e2bbbc1dac0c7d2]) successfully deleted
I20260807 18:14:14.185343 21280 tablet_bootstrap.cc:654] T c5972b7469d745d69312e6788a9c2dc0 P bad08fe563c74ec9bc51bf3159a4ac74: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:14.185673 21279 tablet_bootstrap.cc:654] T c5972b7469d745d69312e6788a9c2dc0 P 067119311361489a9621056d34ddce20: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:14.186632 21281 tablet_bootstrap.cc:492] T c5972b7469d745d69312e6788a9c2dc0 P aa235ba635bf4b5ca63e16ce4aabed61: Bootstrap starting.
I20260807 18:14:14.188695 21281 tablet_bootstrap.cc:654] T c5972b7469d745d69312e6788a9c2dc0 P aa235ba635bf4b5ca63e16ce4aabed61: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:14.189196 21279 tablet_bootstrap.cc:492] T c5972b7469d745d69312e6788a9c2dc0 P 067119311361489a9621056d34ddce20: No bootstrap required, opened a new log
I20260807 18:14:14.189371 21279 ts_tablet_manager.cc:1397] T c5972b7469d745d69312e6788a9c2dc0 P 067119311361489a9621056d34ddce20: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 18:14:14.189826 21280 tablet_bootstrap.cc:492] T c5972b7469d745d69312e6788a9c2dc0 P bad08fe563c74ec9bc51bf3159a4ac74: No bootstrap required, opened a new log
I20260807 18:14:14.190026 21280 ts_tablet_manager.cc:1397] T c5972b7469d745d69312e6788a9c2dc0 P bad08fe563c74ec9bc51bf3159a4ac74: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260807 18:14:14.190250 21279 raft_consensus.cc:348] T c5972b7469d745d69312e6788a9c2dc0 P 067119311361489a9621056d34ddce20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:14.190523 21279 raft_consensus.cc:374] T c5972b7469d745d69312e6788a9c2dc0 P 067119311361489a9621056d34ddce20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:14.190613 21279 raft_consensus.cc:729] T c5972b7469d745d69312e6788a9c2dc0 P 067119311361489a9621056d34ddce20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 067119311361489a9621056d34ddce20, State: Initialized, Role: FOLLOWER
I20260807 18:14:14.190943 21279 consensus_queue.cc:260] T c5972b7469d745d69312e6788a9c2dc0 P 067119311361489a9621056d34ddce20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:14.191533 21279 ts_tablet_manager.cc:1428] T c5972b7469d745d69312e6788a9c2dc0 P 067119311361489a9621056d34ddce20: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 18:14:14.191033 21280 raft_consensus.cc:348] T c5972b7469d745d69312e6788a9c2dc0 P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:14.192409 21280 raft_consensus.cc:374] T c5972b7469d745d69312e6788a9c2dc0 P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:14.192529 21280 raft_consensus.cc:729] T c5972b7469d745d69312e6788a9c2dc0 P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad08fe563c74ec9bc51bf3159a4ac74, State: Initialized, Role: FOLLOWER
I20260807 18:14:14.192821 21280 consensus_queue.cc:260] T c5972b7469d745d69312e6788a9c2dc0 P bad08fe563c74ec9bc51bf3159a4ac74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:14.193712 21280 ts_tablet_manager.cc:1428] T c5972b7469d745d69312e6788a9c2dc0 P bad08fe563c74ec9bc51bf3159a4ac74: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260807 18:14:14.195859 21310 tablet_replica.cc:330] T 44c5af415846459a92649e0a60513356 P aa235ba635bf4b5ca63e16ce4aabed61: stopping tablet replica
I20260807 18:14:14.196135 21310 raft_consensus.cc:2229] T 44c5af415846459a92649e0a60513356 P aa235ba635bf4b5ca63e16ce4aabed61 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:14:14.196910 21310 raft_consensus.cc:2258] T 44c5af415846459a92649e0a60513356 P aa235ba635bf4b5ca63e16ce4aabed61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:14:14.200016 21310 ts_tablet_manager.cc:1905] T 44c5af415846459a92649e0a60513356 P aa235ba635bf4b5ca63e16ce4aabed61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:14:14.205529 21312 ts_tablet_manager.cc:1918] T 44c5af415846459a92649e0a60513356 P bad08fe563c74ec9bc51bf3159a4ac74: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:14:14.205726 21312 log.cc:1196] T 44c5af415846459a92649e0a60513356 P bad08fe563c74ec9bc51bf3159a4ac74: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-2/wal/wals/44c5af415846459a92649e0a60513356
I20260807 18:14:14.206261 21312 ts_tablet_manager.cc:1939] T 44c5af415846459a92649e0a60513356 P bad08fe563c74ec9bc51bf3159a4ac74: Deleting consensus metadata
I20260807 18:14:14.206627 21281 tablet_bootstrap.cc:492] T c5972b7469d745d69312e6788a9c2dc0 P aa235ba635bf4b5ca63e16ce4aabed61: No bootstrap required, opened a new log
I20260807 18:14:14.206951 21281 ts_tablet_manager.cc:1397] T c5972b7469d745d69312e6788a9c2dc0 P aa235ba635bf4b5ca63e16ce4aabed61: Time spent bootstrapping tablet: real 0.020s user 0.008s sys 0.003s
I20260807 18:14:14.208022 21281 raft_consensus.cc:348] T c5972b7469d745d69312e6788a9c2dc0 P aa235ba635bf4b5ca63e16ce4aabed61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:14.208356 21281 raft_consensus.cc:374] T c5972b7469d745d69312e6788a9c2dc0 P aa235ba635bf4b5ca63e16ce4aabed61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:14.208535 21281 raft_consensus.cc:729] T c5972b7469d745d69312e6788a9c2dc0 P aa235ba635bf4b5ca63e16ce4aabed61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa235ba635bf4b5ca63e16ce4aabed61, State: Initialized, Role: FOLLOWER
I20260807 18:14:14.208954 21281 consensus_queue.cc:260] T c5972b7469d745d69312e6788a9c2dc0 P aa235ba635bf4b5ca63e16ce4aabed61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:14.209594 21281 ts_tablet_manager.cc:1428] T c5972b7469d745d69312e6788a9c2dc0 P aa235ba635bf4b5ca63e16ce4aabed61: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 18:14:14.211825 20387 catalog_manager.cc:4841] TS bad08fe563c74ec9bc51bf3159a4ac74 (127.17.110.3:40729): tablet 44c5af415846459a92649e0a60513356 (table test-table [id=82bce5e4ad50411e8e2bbbc1dac0c7d2]) successfully deleted
I20260807 18:14:14.213032 21310 ts_tablet_manager.cc:1918] T 44c5af415846459a92649e0a60513356 P aa235ba635bf4b5ca63e16ce4aabed61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:14:14.213241 21310 log.cc:1196] T 44c5af415846459a92649e0a60513356 P aa235ba635bf4b5ca63e16ce4aabed61: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-1/wal/wals/44c5af415846459a92649e0a60513356
I20260807 18:14:14.213828 21310 ts_tablet_manager.cc:1939] T 44c5af415846459a92649e0a60513356 P aa235ba635bf4b5ca63e16ce4aabed61: Deleting consensus metadata
I20260807 18:14:14.215103 20387 catalog_manager.cc:4841] TS aa235ba635bf4b5ca63e16ce4aabed61 (127.17.110.2:35923): tablet 44c5af415846459a92649e0a60513356 (table test-table [id=82bce5e4ad50411e8e2bbbc1dac0c7d2]) successfully deleted
I20260807 18:14:14.374868 21184 raft_consensus.cc:482] T c5972b7469d745d69312e6788a9c2dc0 P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:14:14.375140 21184 raft_consensus.cc:504] T c5972b7469d745d69312e6788a9c2dc0 P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:14.376003 21184 leader_election.cc:288] T c5972b7469d745d69312e6788a9c2dc0 P bad08fe563c74ec9bc51bf3159a4ac74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 067119311361489a9621056d34ddce20 (127.17.110.1:35891), aa235ba635bf4b5ca63e16ce4aabed61 (127.17.110.2:35923)
I20260807 18:14:14.376708 20638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5972b7469d745d69312e6788a9c2dc0" candidate_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "067119311361489a9621056d34ddce20" is_pre_election: true
I20260807 18:14:14.376715 20840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5972b7469d745d69312e6788a9c2dc0" candidate_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" is_pre_election: true
I20260807 18:14:14.376983 20638 raft_consensus.cc:2454] T c5972b7469d745d69312e6788a9c2dc0 P 067119311361489a9621056d34ddce20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bad08fe563c74ec9bc51bf3159a4ac74 in term 0.
I20260807 18:14:14.377017 20840 raft_consensus.cc:2454] T c5972b7469d745d69312e6788a9c2dc0 P aa235ba635bf4b5ca63e16ce4aabed61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bad08fe563c74ec9bc51bf3159a4ac74 in term 0.
I20260807 18:14:14.377521 20940 leader_election.cc:302] T c5972b7469d745d69312e6788a9c2dc0 P bad08fe563c74ec9bc51bf3159a4ac74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 067119311361489a9621056d34ddce20, bad08fe563c74ec9bc51bf3159a4ac74; no voters:
I20260807 18:14:14.377904 21184 raft_consensus.cc:2781] T c5972b7469d745d69312e6788a9c2dc0 P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:14:14.378031 21184 raft_consensus.cc:482] T c5972b7469d745d69312e6788a9c2dc0 P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:14:14.378145 21184 raft_consensus.cc:3037] T c5972b7469d745d69312e6788a9c2dc0 P bad08fe563c74ec9bc51bf3159a4ac74 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:14.380509 21184 raft_consensus.cc:504] T c5972b7469d745d69312e6788a9c2dc0 P bad08fe563c74ec9bc51bf3159a4ac74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:14.381877 21184 leader_election.cc:288] T c5972b7469d745d69312e6788a9c2dc0 P bad08fe563c74ec9bc51bf3159a4ac74 [CANDIDATE]: Term 1 election: Requested vote from peers 067119311361489a9621056d34ddce20 (127.17.110.1:35891), aa235ba635bf4b5ca63e16ce4aabed61 (127.17.110.2:35923)
I20260807 18:14:14.381888 20638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5972b7469d745d69312e6788a9c2dc0" candidate_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "067119311361489a9621056d34ddce20"
I20260807 18:14:14.382135 20638 raft_consensus.cc:3037] T c5972b7469d745d69312e6788a9c2dc0 P 067119311361489a9621056d34ddce20 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:14.385123 20638 raft_consensus.cc:2454] T c5972b7469d745d69312e6788a9c2dc0 P 067119311361489a9621056d34ddce20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bad08fe563c74ec9bc51bf3159a4ac74 in term 1.
I20260807 18:14:14.385865 20940 leader_election.cc:302] T c5972b7469d745d69312e6788a9c2dc0 P bad08fe563c74ec9bc51bf3159a4ac74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 067119311361489a9621056d34ddce20, bad08fe563c74ec9bc51bf3159a4ac74; no voters:
I20260807 18:14:14.386358 21184 raft_consensus.cc:2781] T c5972b7469d745d69312e6788a9c2dc0 P bad08fe563c74ec9bc51bf3159a4ac74 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:14:14.386426 20840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5972b7469d745d69312e6788a9c2dc0" candidate_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa235ba635bf4b5ca63e16ce4aabed61"
I20260807 18:14:14.386729 21184 raft_consensus.cc:686] T c5972b7469d745d69312e6788a9c2dc0 P bad08fe563c74ec9bc51bf3159a4ac74 [term 1 LEADER]: Becoming Leader. State: Replica: bad08fe563c74ec9bc51bf3159a4ac74, State: Running, Role: LEADER
I20260807 18:14:14.386792 20840 raft_consensus.cc:3037] T c5972b7469d745d69312e6788a9c2dc0 P aa235ba635bf4b5ca63e16ce4aabed61 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:14.387042 21184 consensus_queue.cc:237] T c5972b7469d745d69312e6788a9c2dc0 P bad08fe563c74ec9bc51bf3159a4ac74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } }
I20260807 18:14:14.390537 20840 raft_consensus.cc:2454] T c5972b7469d745d69312e6788a9c2dc0 P aa235ba635bf4b5ca63e16ce4aabed61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bad08fe563c74ec9bc51bf3159a4ac74 in term 1.
I20260807 18:14:14.391103 20408 catalog_manager.cc:5495] T c5972b7469d745d69312e6788a9c2dc0 P bad08fe563c74ec9bc51bf3159a4ac74 reported cstate change: term changed from 0 to 1, leader changed from <none> to bad08fe563c74ec9bc51bf3159a4ac74 (127.17.110.3). New cstate: current_term: 1 leader_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bad08fe563c74ec9bc51bf3159a4ac74" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 40729 } health_report { overall_health: HEALTHY } } }
I20260807 18:14:14.421919 21299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:14.414556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36120 (local address 127.17.110.3:40729)
0807 18:14:14.414760 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:14.414766 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:14.415014 (+ 248us) server_negotiation.cc:407] Connection header received
0807 18:14:14.415125 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:14.415131 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:14.415230 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:14.415336 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:14:14.415935 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:14.417191 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:14.419445 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:14.420217 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:14.420372 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:14.420954 (+ 582us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:14:14.421156 (+ 202us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:14:14.421268 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:14.421468 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 18:14:14.421571 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 18:14:14.434917 20840 raft_consensus.cc:1261] T c5972b7469d745d69312e6788a9c2dc0 P aa235ba635bf4b5ca63e16ce4aabed61 [term 1 FOLLOWER]: Refusing update from remote peer bad08fe563c74ec9bc51bf3159a4ac74: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:14:14.434914 20638 raft_consensus.cc:1261] T c5972b7469d745d69312e6788a9c2dc0 P 067119311361489a9621056d34ddce20 [term 1 FOLLOWER]: Refusing update from remote peer bad08fe563c74ec9bc51bf3159a4ac74: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:14:14.435745 21248 consensus_queue.cc:1035] T c5972b7469d745d69312e6788a9c2dc0 P bad08fe563c74ec9bc51bf3159a4ac74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa235ba635bf4b5ca63e16ce4aabed61" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:14:14.436406 21184 consensus_queue.cc:1035] T c5972b7469d745d69312e6788a9c2dc0 P bad08fe563c74ec9bc51bf3159a4ac74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "067119311361489a9621056d34ddce20" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 35891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 18:14:14.468809 21234 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 18:14:16.075500 20408 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:14:16.075860 20408 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:14:16.080869 20408 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d176896ac4b84f47add8a59c25daceec: Sending DeleteTablet for 3 replicas of tablet c5972b7469d745d69312e6788a9c2dc0
I20260807 18:14:16.081933 20605 tablet_service.cc:1514] Processing DeleteTablet for tablet c5972b7469d745d69312e6788a9c2dc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:14:16 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:56934
I20260807 18:14:16.082017 20813 tablet_service.cc:1514] Processing DeleteTablet for tablet c5972b7469d745d69312e6788a9c2dc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:14:16 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:59840
I20260807 18:14:16.082481 21002 tablet_service.cc:1514] Processing DeleteTablet for tablet c5972b7469d745d69312e6788a9c2dc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:14:16 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:56962
I20260807 18:14:16.083032 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 20511
I20260807 18:14:16.098358 21358 tablet_replica.cc:330] T c5972b7469d745d69312e6788a9c2dc0 P aa235ba635bf4b5ca63e16ce4aabed61: stopping tablet replica
I20260807 18:14:16.098688 21358 raft_consensus.cc:2229] T c5972b7469d745d69312e6788a9c2dc0 P aa235ba635bf4b5ca63e16ce4aabed61 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:14:16.099026 21358 raft_consensus.cc:2258] T c5972b7469d745d69312e6788a9c2dc0 P aa235ba635bf4b5ca63e16ce4aabed61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:14:16.101068 21358 ts_tablet_manager.cc:1905] T c5972b7469d745d69312e6788a9c2dc0 P aa235ba635bf4b5ca63e16ce4aabed61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:14:16.102078 21359 tablet_replica.cc:330] T c5972b7469d745d69312e6788a9c2dc0 P bad08fe563c74ec9bc51bf3159a4ac74: stopping tablet replica
I20260807 18:14:16.102321 21359 raft_consensus.cc:2229] T c5972b7469d745d69312e6788a9c2dc0 P bad08fe563c74ec9bc51bf3159a4ac74 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:14:16.102759 21355 tablet_replica.cc:330] T c5972b7469d745d69312e6788a9c2dc0 P 067119311361489a9621056d34ddce20: stopping tablet replica
I20260807 18:14:16.102905 21359 raft_consensus.cc:2258] T c5972b7469d745d69312e6788a9c2dc0 P bad08fe563c74ec9bc51bf3159a4ac74 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:14:16.102988 21355 raft_consensus.cc:2229] T c5972b7469d745d69312e6788a9c2dc0 P 067119311361489a9621056d34ddce20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:14:16.103250 21355 raft_consensus.cc:2258] T c5972b7469d745d69312e6788a9c2dc0 P 067119311361489a9621056d34ddce20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:14:16.104214 21359 ts_tablet_manager.cc:1905] T c5972b7469d745d69312e6788a9c2dc0 P bad08fe563c74ec9bc51bf3159a4ac74: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:14:16.104336 21355 ts_tablet_manager.cc:1905] T c5972b7469d745d69312e6788a9c2dc0 P 067119311361489a9621056d34ddce20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:14:16.108700 21358 ts_tablet_manager.cc:1918] T c5972b7469d745d69312e6788a9c2dc0 P aa235ba635bf4b5ca63e16ce4aabed61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:14:16.108881 21358 log.cc:1196] T c5972b7469d745d69312e6788a9c2dc0 P aa235ba635bf4b5ca63e16ce4aabed61: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-1/wal/wals/c5972b7469d745d69312e6788a9c2dc0
I20260807 18:14:16.109568 21358 ts_tablet_manager.cc:1939] T c5972b7469d745d69312e6788a9c2dc0 P aa235ba635bf4b5ca63e16ce4aabed61: Deleting consensus metadata
I20260807 18:14:16.110925 20387 catalog_manager.cc:4841] TS aa235ba635bf4b5ca63e16ce4aabed61 (127.17.110.2:35923): tablet c5972b7469d745d69312e6788a9c2dc0 (table test-table [id=1a763edff4bf4de5a8b0c2234fe116f1]) successfully deleted
I20260807 18:14:16.111697 21359 ts_tablet_manager.cc:1918] T c5972b7469d745d69312e6788a9c2dc0 P bad08fe563c74ec9bc51bf3159a4ac74: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:14:16.111869 21359 log.cc:1196] T c5972b7469d745d69312e6788a9c2dc0 P bad08fe563c74ec9bc51bf3159a4ac74: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-2/wal/wals/c5972b7469d745d69312e6788a9c2dc0
I20260807 18:14:16.112321 21359 ts_tablet_manager.cc:1939] T c5972b7469d745d69312e6788a9c2dc0 P bad08fe563c74ec9bc51bf3159a4ac74: Deleting consensus metadata
I20260807 18:14:16.112598 21355 ts_tablet_manager.cc:1918] T c5972b7469d745d69312e6788a9c2dc0 P 067119311361489a9621056d34ddce20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:14:16.112761 21355 log.cc:1196] T c5972b7469d745d69312e6788a9c2dc0 P 067119311361489a9621056d34ddce20: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126429568977-17848-0/minicluster-data/ts-0/wal/wals/c5972b7469d745d69312e6788a9c2dc0
I20260807 18:14:16.113188 21355 ts_tablet_manager.cc:1939] T c5972b7469d745d69312e6788a9c2dc0 P 067119311361489a9621056d34ddce20: Deleting consensus metadata
I20260807 18:14:16.113451 20387 catalog_manager.cc:4841] TS bad08fe563c74ec9bc51bf3159a4ac74 (127.17.110.3:40729): tablet c5972b7469d745d69312e6788a9c2dc0 (table test-table [id=1a763edff4bf4de5a8b0c2234fe116f1]) successfully deleted
I20260807 18:14:16.114526 20388 catalog_manager.cc:4841] TS 067119311361489a9621056d34ddce20 (127.17.110.1:35891): tablet c5972b7469d745d69312e6788a9c2dc0 (table test-table [id=1a763edff4bf4de5a8b0c2234fe116f1]) successfully deleted
Aug 07 18:14:16 dist-test-slave-39l8 krb5kdc[20345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126453, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.110.1@KRBTEST.COM
I20260807 18:14:16.127034 21361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:16.084699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46420 (local address 127.17.110.1:35891)
0807 18:14:16.100012 (+ 15313us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:16.100022 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:16.102933 (+ 2911us) server_negotiation.cc:407] Connection header received
0807 18:14:16.103134 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:16.103141 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:16.103245 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:16.103491 (+ 246us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:16.105725 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:16.107167 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:16.109371 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:16.110177 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:16.110322 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:16.119436 (+ 9114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:16.119455 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:16.119473 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:16.119523 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:16.123341 (+ 3818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:16.124394 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:16.124404 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:16.124407 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:16.124596 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:16.125170 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:16.125176 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:16.125178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:16.125625 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:16.125925 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:16.126275 (+ 350us) server_negotiation.cc:299] Negotiation successful
0807 18:14:16.126514 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15065,"thread_start_us":14890,"threads_started":1}
W20260807 18:14:16.127859 20707 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46420
W20260807 18:14:16.128657 17848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 20511: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:14:16.128950 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 20511
I20260807 18:14:16.170205 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 20718
Aug 07 18:14:16 dist-test-slave-39l8 krb5kdc[20345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126453, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.110.2@KRBTEST.COM
I20260807 18:14:16.188086 21362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:16.171083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56884 (local address 127.17.110.2:35923)
0807 18:14:16.171763 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:16.171768 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:16.171787 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:14:16.171844 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:16.171848 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:16.171916 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:16.172028 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:16.173375 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:16.174162 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:16.175597 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:16.176032 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:16.176143 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:16.180874 (+ 4731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:16.180892 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:16.180913 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:16.180962 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:16.184669 (+ 3707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:16.185511 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:16.185520 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:16.185523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:16.185721 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:16.186191 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:16.186196 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:16.186198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:16.186602 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:16.186921 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:16.187385 (+ 464us) server_negotiation.cc:299] Negotiation successful
0807 18:14:16.187631 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":323,"threads_started":1}
W20260807 18:14:16.189059 20898 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56884
W20260807 18:14:16.189922 17848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 20718: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:14:16.190155 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 20718
I20260807 18:14:16.228861 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 20915
Aug 07 18:14:16 dist-test-slave-39l8 krb5kdc[20345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126453, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.110.3@KRBTEST.COM
I20260807 18:14:16.247509 21363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:16.229797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36130 (local address 127.17.110.3:40729)
0807 18:14:16.230431 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:16.230436 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:16.230455 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:14:16.230516 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:16.230520 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:16.230601 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:16.230795 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:16.232346 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:16.233212 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:16.234630 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:16.235343 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:16.235502 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:16.240517 (+ 5015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:16.240538 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:16.240558 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:16.240606 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:16.244141 (+ 3535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:16.245062 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:16.245073 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:16.245077 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:16.245252 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:16.245710 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:16.245714 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:16.245716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:16.246136 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:16.246390 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:16.246913 (+ 523us) server_negotiation.cc:299] Negotiation successful
0807 18:14:16.247190 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":333,"threads_started":1}
W20260807 18:14:16.248283 21093 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36130
W20260807 18:14:16.248998 17848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 20915: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:14:16.249187 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 20915
I20260807 18:14:16.290733 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 20363
I20260807 18:14:16.292052 20467 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:14:16.416832 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 20363
2026-08-07T18:14:16Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9941 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:14:16 dist-test-slave-39l8 krb5kdc[21367](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:14:16 dist-test-slave-39l8 krb5kdc[21367](info): set up 2 sockets
Aug 07 18:14:16 dist-test-slave-39l8 krb5kdc[21367](info): commencing operation
krb5kdc: starting...
W20260807 18:14:17.778277 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.306s user 0.005s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:14:17 dist-test-slave-39l8 krb5kdc[21367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:14:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:14:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.110.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
I20260807 18:14:18.035306 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:33509
--webserver_interface=127.17.110.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:42679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:33509
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:14:18.416293 21386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:18.416623 21386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:18.416786 21386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:18.424979 21386 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:14:18.425160 21386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:18.425248 21386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:18.425384 21386 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:14:18.425483 21386 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:14:18.437155 21386 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:42679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:33509
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:33509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:18.438722 21386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:18.440289 21386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:18.454193 21395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:18.454418 21392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:18.454931 21399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:18.455271 21386 server_base.cc:1034] running on GCE node
I20260807 18:14:18.456161 21386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:18.457795 21386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:18.459080 21386 hybrid_clock.cc:648] HybridClock initialized: now 1786126458459012 us; error 59 us; skew 500 ppm
I20260807 18:14:18.459487 21386 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:18 dist-test-slave-39l8 krb5kdc[21367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126458, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:18.462966 21386 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:14:18.464673 21386 webserver.cc:460] Webserver started at http://127.17.110.62:34923/ using document root <none> and password file <none>
I20260807 18:14:18.465433 21386 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:18.465583 21386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:18.465929 21386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:18.469363 21386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "deb7ad6b86944400b9abe4901ea198e2"
format_stamp: "Formatted at 2026-08-07 18:14:18 on dist-test-slave-39l8"
server_key: "3c386fc9d670b6375d2b202f476345c7"
server_key_iv: "3f4477a456bbd7969b0c878b2cc83d32"
server_key_version: "encryptionkey@0"
I20260807 18:14:18.470261 21386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "deb7ad6b86944400b9abe4901ea198e2"
format_stamp: "Formatted at 2026-08-07 18:14:18 on dist-test-slave-39l8"
server_key: "3c386fc9d670b6375d2b202f476345c7"
server_key_iv: "3f4477a456bbd7969b0c878b2cc83d32"
server_key_version: "encryptionkey@0"
I20260807 18:14:18.476572 21386 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260807 18:14:18.481047 21408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:18.482618 21386 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:14:18.482888 21386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "deb7ad6b86944400b9abe4901ea198e2"
format_stamp: "Formatted at 2026-08-07 18:14:18 on dist-test-slave-39l8"
server_key: "3c386fc9d670b6375d2b202f476345c7"
server_key_iv: "3f4477a456bbd7969b0c878b2cc83d32"
server_key_version: "encryptionkey@0"
I20260807 18:14:18.483160 21386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:18.497447 21386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:18.502929 21386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:18.503273 21386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:18.534323 21386 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:33509
I20260807 18:14:18.534333 21479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:33509 every 8 connection(s)
I20260807 18:14:18.536753 21386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:14:18.541736 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 21386
I20260807 18:14:18.542045 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:14:18.542791 21481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:18.542908 17848 external_mini_cluster.cc:1428] Setting key 161245e3fc5a9c1d77010a056d496fed
I20260807 18:14:18.562273 21481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P deb7ad6b86944400b9abe4901ea198e2: Bootstrap starting.
Aug 07 18:14:18 dist-test-slave-39l8 krb5kdc[21367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:18.568049 21481 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P deb7ad6b86944400b9abe4901ea198e2: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:18.569873 21481 log.cc:824] T 00000000000000000000000000000000 P deb7ad6b86944400b9abe4901ea198e2: Log is configured to *not* fsync() on all Append() calls
I20260807 18:14:18.574496 21481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P deb7ad6b86944400b9abe4901ea198e2: No bootstrap required, opened a new log
I20260807 18:14:18.579550 21484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:18.545469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56182 (local address 127.17.110.62:33509)
0807 18:14:18.547016 (+ 1547us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:18.547030 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:18.547085 (+ 55us) server_negotiation.cc:407] Connection header received
0807 18:14:18.549944 (+ 2859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:18.549986 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:18.550659 (+ 673us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:18.551258 (+ 599us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:18.553797 (+ 2539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:18.555492 (+ 1695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:18.557986 (+ 2494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:18.558970 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:18.559240 (+ 270us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:18.568454 (+ 9214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:18.568466 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:18.568484 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:18.568538 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:18.573349 (+ 4811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:18.574439 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:18.574447 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:18.574455 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:18.574820 (+ 365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:18.575359 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:18.575364 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:18.575367 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:18.576410 (+ 1043us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:18.576718 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:18.577275 (+ 557us) server_negotiation.cc:299] Negotiation successful
0807 18:14:18.577821 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1282,"thread_start_us":885,"threads_started":1}
I20260807 18:14:18.587296 21481 raft_consensus.cc:348] T 00000000000000000000000000000000 P deb7ad6b86944400b9abe4901ea198e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deb7ad6b86944400b9abe4901ea198e2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 33509 } }
I20260807 18:14:18.587805 21481 raft_consensus.cc:374] T 00000000000000000000000000000000 P deb7ad6b86944400b9abe4901ea198e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:18.587949 21481 raft_consensus.cc:729] T 00000000000000000000000000000000 P deb7ad6b86944400b9abe4901ea198e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: deb7ad6b86944400b9abe4901ea198e2, State: Initialized, Role: FOLLOWER
I20260807 18:14:18.588769 21481 consensus_queue.cc:260] T 00000000000000000000000000000000 P deb7ad6b86944400b9abe4901ea198e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deb7ad6b86944400b9abe4901ea198e2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 33509 } }
I20260807 18:14:18.589063 21481 raft_consensus.cc:388] T 00000000000000000000000000000000 P deb7ad6b86944400b9abe4901ea198e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:14:18.589229 21481 raft_consensus.cc:482] T 00000000000000000000000000000000 P deb7ad6b86944400b9abe4901ea198e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:14:18.589450 21481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P deb7ad6b86944400b9abe4901ea198e2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:18.591987 21481 raft_consensus.cc:504] T 00000000000000000000000000000000 P deb7ad6b86944400b9abe4901ea198e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deb7ad6b86944400b9abe4901ea198e2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 33509 } }
I20260807 18:14:18.592592 21481 leader_election.cc:302] T 00000000000000000000000000000000 P deb7ad6b86944400b9abe4901ea198e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: deb7ad6b86944400b9abe4901ea198e2; no voters:
I20260807 18:14:18.593449 21481 leader_election.cc:288] T 00000000000000000000000000000000 P deb7ad6b86944400b9abe4901ea198e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:14:18.593565 21487 raft_consensus.cc:2781] T 00000000000000000000000000000000 P deb7ad6b86944400b9abe4901ea198e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:14:18.594377 21487 raft_consensus.cc:686] T 00000000000000000000000000000000 P deb7ad6b86944400b9abe4901ea198e2 [term 1 LEADER]: Becoming Leader. State: Replica: deb7ad6b86944400b9abe4901ea198e2, State: Running, Role: LEADER
I20260807 18:14:18.595042 21487 consensus_queue.cc:237] T 00000000000000000000000000000000 P deb7ad6b86944400b9abe4901ea198e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deb7ad6b86944400b9abe4901ea198e2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 33509 } }
I20260807 18:14:18.596164 21481 sys_catalog.cc:564] T 00000000000000000000000000000000 P deb7ad6b86944400b9abe4901ea198e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:14:18.598470 21488 sys_catalog.cc:455] T 00000000000000000000000000000000 P deb7ad6b86944400b9abe4901ea198e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "deb7ad6b86944400b9abe4901ea198e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deb7ad6b86944400b9abe4901ea198e2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 33509 } } }
I20260807 18:14:18.599278 21488 sys_catalog.cc:458] T 00000000000000000000000000000000 P deb7ad6b86944400b9abe4901ea198e2 [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:18.600281 21497 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:14:18.600172 21489 sys_catalog.cc:455] T 00000000000000000000000000000000 P deb7ad6b86944400b9abe4901ea198e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader deb7ad6b86944400b9abe4901ea198e2. Latest consensus state: current_term: 1 leader_uuid: "deb7ad6b86944400b9abe4901ea198e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deb7ad6b86944400b9abe4901ea198e2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 33509 } } }
I20260807 18:14:18.600592 21489 sys_catalog.cc:458] T 00000000000000000000000000000000 P deb7ad6b86944400b9abe4901ea198e2 [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:18.608826 21497 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:14:18.620409 21497 catalog_manager.cc:1347] Generated new cluster ID: d298b4448cb8414d8cd238093bb41133
I20260807 18:14:18.620571 21497 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:14:18.644665 21497 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:14:18.646793 21497 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:14:18.654990 21497 catalog_manager.cc:5868] T 00000000000000000000000000000000 P deb7ad6b86944400b9abe4901ea198e2: Generated new TSK 0
I20260807 18:14:18.655984 21497 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.110.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
I20260807 18:14:18.715440 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.1:0
--local_ip_for_outbound_sockets=127.17.110.1
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:33509
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:42679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:14:19.094592 21517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:19.095074 21517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:19.095276 21517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:19.103598 21517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:19.103776 21517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:19.103974 21517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.1
I20260807 18:14:19.116503 21517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:42679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:33509
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.1
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:19.118063 21517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:19.119580 21517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:19.135679 21526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:19.135749 21527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:19.135969 21530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:19.137056 21517 server_base.cc:1034] running on GCE node
I20260807 18:14:19.137792 21517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:19.139019 21517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:19.140275 21517 hybrid_clock.cc:648] HybridClock initialized: now 1786126459140210 us; error 60 us; skew 500 ppm
I20260807 18:14:19.140678 21517 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:19 dist-test-slave-39l8 krb5kdc[21367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126459, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:19.144903 21517 init.cc:377] Logged in from keytab as kudu/127.17.110.1@KRBTEST.COM (short username kudu)
I20260807 18:14:19.146876 21517 webserver.cc:460] Webserver started at http://127.17.110.1:39485/ using document root <none> and password file <none>
I20260807 18:14:19.147742 21517 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:19.147891 21517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:19.148257 21517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:19.151784 21517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/data/instance:
uuid: "5508eb5078cd4e589f542983fd4e4924"
format_stamp: "Formatted at 2026-08-07 18:14:19 on dist-test-slave-39l8"
server_key: "8f4e7b4baeacea09ec597f57e0b4b30f"
server_key_iv: "258cb3488077346282851e7c6c987d48"
server_key_version: "encryptionkey@0"
I20260807 18:14:19.152729 21517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance:
uuid: "5508eb5078cd4e589f542983fd4e4924"
format_stamp: "Formatted at 2026-08-07 18:14:19 on dist-test-slave-39l8"
server_key: "8f4e7b4baeacea09ec597f57e0b4b30f"
server_key_iv: "258cb3488077346282851e7c6c987d48"
server_key_version: "encryptionkey@0"
I20260807 18:14:19.159138 21517 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 18:14:19.163483 21537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:19.165099 21517 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:14:19.165339 21517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/wal
uuid: "5508eb5078cd4e589f542983fd4e4924"
format_stamp: "Formatted at 2026-08-07 18:14:19 on dist-test-slave-39l8"
server_key: "8f4e7b4baeacea09ec597f57e0b4b30f"
server_key_iv: "258cb3488077346282851e7c6c987d48"
server_key_version: "encryptionkey@0"
I20260807 18:14:19.165560 21517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:19.193789 21517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:19.198874 21517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:19.199208 21517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:19.201129 21517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:14:19.202749 21517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:14:19.202867 21517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:19.203042 21517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:14:19.203119 21517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:19.240978 21517 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.1:34635
I20260807 18:14:19.241019 21696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.1:34635 every 8 connection(s)
I20260807 18:14:19.242563 21517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
I20260807 18:14:19.243294 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 21517
I20260807 18:14:19.243558 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance
I20260807 18:14:19.244467 17848 external_mini_cluster.cc:1428] Setting key a56451618486c023c673557dca9e9925
Aug 07 18:14:19 dist-test-slave-39l8 krb5kdc[21367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126459, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.110.2@KRBTEST.COM; defaulting to no policy
I20260807 18:14:19.268602 21699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:19.246016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:51455 (local address 127.17.110.62:33509)
0807 18:14:19.246679 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:19.246685 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:19.247865 (+ 1180us) server_negotiation.cc:407] Connection header received
0807 18:14:19.250164 (+ 2299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:19.250173 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:19.250289 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:19.250423 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:19.252223 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:19.253452 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:19.256116 (+ 2664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:19.256563 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:19.256677 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:19.261714 (+ 5037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:19.261721 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:19.261725 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:19.261768 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:19.264989 (+ 3221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:19.265981 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:19.265986 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:19.265988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:19.266123 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:19.266619 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:19.266623 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:19.266625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:19.266990 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:19.267161 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:19.267955 (+ 794us) server_negotiation.cc:299] Negotiation successful
0807 18:14:19.268205 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":413,"threads_started":1}
I20260807 18:14:19.269716 21700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:19.246676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:33509 (local address 127.17.110.1:51455)
0807 18:14:19.247662 (+ 986us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:19.247726 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:19.249832 (+ 2106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:19.250676 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:19.250686 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:19.251344 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:19.251995 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:19.252029 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:19.253659 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:19.253663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:19.255906 (+ 2243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:19.255929 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:19.256854 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:19.256864 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:19.257236 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:19.258120 (+ 884us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:19.258160 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:19.261357 (+ 3197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:19.265189 (+ 3832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:19.265200 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:19.265233 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:19.265826 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:19.266250 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:19.266256 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:19.266260 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:19.266474 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:19.267146 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:19.267155 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:19.267543 (+ 388us) client_negotiation.cc:769] Sending connection context
0807 18:14:19.267828 (+ 285us) client_negotiation.cc:240] Negotiation successful
0807 18:14:19.268434 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":390,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.2@KRBTEST.COM" created.
I20260807 18:14:19.271883 21697 heartbeater.cc:347] Connected to a master server at 127.17.110.62:33509
I20260807 18:14:19.272285 21697 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:19.273141 21697 heartbeater.cc:511] Master 127.17.110.62:33509 requested a full tablet report, sending...
I20260807 18:14:19.275951 21435 ts_manager.cc:194] Registered new tserver with Master: 5508eb5078cd4e589f542983fd4e4924 (127.17.110.1:34635)
I20260807 18:14:19.278434 21435 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.1@KRBTEST.COM'} at 127.17.110.1:51455
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
I20260807 18:14:19.300143 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.2:0
--local_ip_for_outbound_sockets=127.17.110.2
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:33509
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:42679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:14:19.666940 21704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:19.667387 21704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:19.667557 21704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:19.675545 21704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:19.675731 21704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:19.675922 21704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.2
I20260807 18:14:19.686421 21704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:42679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:33509
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.2
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:19.687965 21704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:19.689424 21704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:19.703753 21712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:19.703982 21710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:19.703770 21714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:19.705204 21704 server_base.cc:1034] running on GCE node
I20260807 18:14:19.705955 21704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:19.707109 21704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:19.708343 21704 hybrid_clock.cc:648] HybridClock initialized: now 1786126459708283 us; error 44 us; skew 500 ppm
I20260807 18:14:19.708726 21704 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:19 dist-test-slave-39l8 krb5kdc[21367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126459, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:19.712551 21704 init.cc:377] Logged in from keytab as kudu/127.17.110.2@KRBTEST.COM (short username kudu)
I20260807 18:14:19.714303 21704 webserver.cc:460] Webserver started at http://127.17.110.2:45761/ using document root <none> and password file <none>
I20260807 18:14:19.715229 21704 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:19.715394 21704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:19.715770 21704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:19.719357 21704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/data/instance:
uuid: "96db19876b924e699cd86e8f18e2b496"
format_stamp: "Formatted at 2026-08-07 18:14:19 on dist-test-slave-39l8"
server_key: "ecc8831498631e6de9617b5362807f7c"
server_key_iv: "0ca380f03ee3277f1f0fb4fd2258647e"
server_key_version: "encryptionkey@0"
I20260807 18:14:19.720302 21704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance:
uuid: "96db19876b924e699cd86e8f18e2b496"
format_stamp: "Formatted at 2026-08-07 18:14:19 on dist-test-slave-39l8"
server_key: "ecc8831498631e6de9617b5362807f7c"
server_key_iv: "0ca380f03ee3277f1f0fb4fd2258647e"
server_key_version: "encryptionkey@0"
I20260807 18:14:19.726599 21704 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 18:14:19.730921 21723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:19.732523 21704 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:14:19.732779 21704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/wal
uuid: "96db19876b924e699cd86e8f18e2b496"
format_stamp: "Formatted at 2026-08-07 18:14:19 on dist-test-slave-39l8"
server_key: "ecc8831498631e6de9617b5362807f7c"
server_key_iv: "0ca380f03ee3277f1f0fb4fd2258647e"
server_key_version: "encryptionkey@0"
I20260807 18:14:19.733017 21704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:19.752667 21704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:19.757951 21704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:19.758297 21704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:19.759443 21704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:14:19.760876 21704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:14:19.761003 21704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:19.761164 21704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:14:19.761240 21704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:19.798189 21704 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.2:36045
I20260807 18:14:19.798244 21873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.2:36045 every 8 connection(s)
I20260807 18:14:19.799762 21704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
I20260807 18:14:19.805958 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 21704
I20260807 18:14:19.806181 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance
I20260807 18:14:19.806944 17848 external_mini_cluster.cc:1428] Setting key c6e2a93eb2493447c34b517948aa5556
Aug 07 18:14:19 dist-test-slave-39l8 krb5kdc[21367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126459, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:19.823783 21877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:19.802468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:33043 (local address 127.17.110.62:33509)
0807 18:14:19.802975 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:19.802981 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:19.804060 (+ 1079us) server_negotiation.cc:407] Connection header received
0807 18:14:19.806333 (+ 2273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:19.806341 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:19.806461 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:19.806648 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:19.808599 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:19.809539 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:19.811473 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:19.812274 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:19.812444 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:19.816531 (+ 4087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:19.816536 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:19.816540 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:19.816583 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:19.819567 (+ 2984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:19.820773 (+ 1206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:19.820778 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:19.820781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:19.820941 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:19.821554 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:19.821558 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:19.821560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:19.821885 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:19.822018 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:19.823204 (+ 1186us) server_negotiation.cc:299] Negotiation successful
0807 18:14:19.823427 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":274,"threads_started":1}
I20260807 18:14:19.825436 21878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:19.803080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:33509 (local address 127.17.110.2:33043)
0807 18:14:19.803853 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:19.803906 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:19.805842 (+ 1936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:19.806933 (+ 1091us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:19.806942 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:19.807646 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:19.808396 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:19.808415 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:19.809770 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:19.809775 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:19.811291 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:19.811307 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:19.812539 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:19.812545 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:19.812772 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:19.813563 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:19.813588 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:19.816202 (+ 2614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:19.819769 (+ 3567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:19.819783 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:19.819809 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:19.820560 (+ 751us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:19.821087 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:19.821095 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:19.821101 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:19.821401 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:19.822036 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:19.822048 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:19.822616 (+ 568us) client_negotiation.cc:769] Sending connection context
0807 18:14:19.823050 (+ 434us) client_negotiation.cc:240] Negotiation successful
0807 18:14:19.823749 (+ 699us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":281,"threads_started":1}
I20260807 18:14:19.827379 21874 heartbeater.cc:347] Connected to a master server at 127.17.110.62:33509
I20260807 18:14:19.827764 21874 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:19.828624 21874 heartbeater.cc:511] Master 127.17.110.62:33509 requested a full tablet report, sending...
I20260807 18:14:19.830309 21435 ts_manager.cc:194] Registered new tserver with Master: 96db19876b924e699cd86e8f18e2b496 (127.17.110.2:36045)
WARNING: no policy specified for kudu/127.17.110.3@KRBTEST.COM; defaulting to no policy
I20260807 18:14:19.831422 21435 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.2@KRBTEST.COM'} at 127.17.110.2:33043
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
I20260807 18:14:19.862792 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.3:0
--local_ip_for_outbound_sockets=127.17.110.3
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:33509
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:42679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:14:20.229943 21881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:20.230345 21881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:20.230526 21881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:20.238627 21881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:20.238843 21881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:20.239027 21881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.3
I20260807 18:14:20.252454 21881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:42679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:33509
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.3
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:20.253878 21881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:20.255460 21881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:20.271277 21894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:20.271253 21887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:20.272046 21886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:20.272130 21881 server_base.cc:1034] running on GCE node
I20260807 18:14:20.272917 21881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:20.273980 21881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:20.275256 21881 hybrid_clock.cc:648] HybridClock initialized: now 1786126460275234 us; error 85 us; skew 500 ppm
I20260807 18:14:20.275655 21881 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:20 dist-test-slave-39l8 krb5kdc[21367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126460, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:20.279510 21881 init.cc:377] Logged in from keytab as kudu/127.17.110.3@KRBTEST.COM (short username kudu)
I20260807 18:14:20.281284 21881 webserver.cc:460] Webserver started at http://127.17.110.3:35721/ using document root <none> and password file <none>
I20260807 18:14:20.281874 21697 heartbeater.cc:503] Master 127.17.110.62:33509 was elected leader, sending a full tablet report...
I20260807 18:14:20.282147 21881 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:20.282310 21881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:20.282801 21881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:20.286198 21881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/data/instance:
uuid: "9249fd0d877a48ebbee8ee95dbd96f76"
format_stamp: "Formatted at 2026-08-07 18:14:20 on dist-test-slave-39l8"
server_key: "0e3aa7832a2aabc40cc1ee87605f031c"
server_key_iv: "4f80a59fad4ede135286c29aad134bff"
server_key_version: "encryptionkey@0"
I20260807 18:14:20.287164 21881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance:
uuid: "9249fd0d877a48ebbee8ee95dbd96f76"
format_stamp: "Formatted at 2026-08-07 18:14:20 on dist-test-slave-39l8"
server_key: "0e3aa7832a2aabc40cc1ee87605f031c"
server_key_iv: "4f80a59fad4ede135286c29aad134bff"
server_key_version: "encryptionkey@0"
I20260807 18:14:20.293207 21881 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 18:14:20.297660 21901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:20.299566 21881 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:14:20.299787 21881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/wal
uuid: "9249fd0d877a48ebbee8ee95dbd96f76"
format_stamp: "Formatted at 2026-08-07 18:14:20 on dist-test-slave-39l8"
server_key: "0e3aa7832a2aabc40cc1ee87605f031c"
server_key_iv: "4f80a59fad4ede135286c29aad134bff"
server_key_version: "encryptionkey@0"
I20260807 18:14:20.300055 21881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:20.337668 21881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:20.342823 21881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:20.343165 21881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:20.344336 21881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:14:20.345906 21881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:14:20.345996 21881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:20.346177 21881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:14:20.346235 21881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:20.384902 21881 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.3:45213
I20260807 18:14:20.385154 22049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.3:45213 every 8 connection(s)
I20260807 18:14:20.386348 21881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
I20260807 18:14:20.389353 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 21881
I20260807 18:14:20.389576 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance
I20260807 18:14:20.390169 17848 external_mini_cluster.cc:1428] Setting key 24108da9000081ee26ebc4ad4a752936
Aug 07 18:14:20 dist-test-slave-39l8 krb5kdc[21367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126460, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:20.411062 22053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:20.389422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:50591 (local address 127.17.110.62:33509)
0807 18:14:20.390041 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:20.390048 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:20.391229 (+ 1181us) server_negotiation.cc:407] Connection header received
0807 18:14:20.393774 (+ 2545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:20.393779 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:20.393884 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:20.394019 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:20.395861 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:20.396791 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:20.398587 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:20.399097 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:20.399262 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:20.403507 (+ 4245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:20.403514 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:20.403519 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:20.403561 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:20.407415 (+ 3854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:20.408296 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:20.408302 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:20.408306 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:20.408452 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:20.408989 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:20.408994 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:20.408997 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:20.409317 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:20.409509 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:20.410364 (+ 855us) server_negotiation.cc:299] Negotiation successful
0807 18:14:20.410613 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":330,"threads_started":1}
I20260807 18:14:20.412180 22056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:20.390054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:33509 (local address 127.17.110.3:50591)
0807 18:14:20.391007 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:20.391082 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:20.393378 (+ 2296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:20.394309 (+ 931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:20.394318 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:20.394992 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:20.395650 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:20.395676 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:20.396972 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:20.396976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:20.398414 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:20.398429 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:20.399374 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:20.399381 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:20.399654 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:20.400461 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:20.400489 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:20.403187 (+ 2698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:20.407588 (+ 4401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:20.407599 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:20.407628 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:20.408145 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:20.408577 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:20.408582 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:20.408586 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:20.408839 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:20.409464 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:20.409474 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:20.409934 (+ 460us) client_negotiation.cc:769] Sending connection context
0807 18:14:20.410242 (+ 308us) client_negotiation.cc:240] Negotiation successful
0807 18:14:20.410912 (+ 670us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":367,"threads_started":1}
I20260807 18:14:20.413892 22051 heartbeater.cc:347] Connected to a master server at 127.17.110.62:33509
I20260807 18:14:20.414232 22051 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:20.415074 22051 heartbeater.cc:511] Master 127.17.110.62:33509 requested a full tablet report, sending...
I20260807 18:14:20.416683 21435 ts_manager.cc:194] Registered new tserver with Master: 9249fd0d877a48ebbee8ee95dbd96f76 (127.17.110.3:45213)
I20260807 18:14:20.417547 21435 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.3@KRBTEST.COM'} at 127.17.110.3:50591
I20260807 18:14:20.420032 17848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 18:14:20.463948 22053 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:14:20.460909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56190 (local address 127.17.110.62:33509)
0807 18:14:20.461283 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:20.461294 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:20.461601 (+ 307us) server_negotiation.cc:407] Connection header received
0807 18:14:20.461858 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:20.461863 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:20.461937 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:20.462070 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:20.463667 (+ 1597us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56190: BlockingRecv error: recv got EOF from 127.0.0.1:56190 (error 108)
Metrics: {"server-negotiator.queue_time_us":189}
I20260807 18:14:20.468781 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 21517
W20260807 18:14:20.472543 17848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 21517: Not authorized: Client connection negotiation failed: client connection to 127.17.110.1:34635: server requires authentication, but client does not have Kerberos credentials available
I20260807 18:14:20.472766 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 21517
I20260807 18:14:20.498296 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 21704
W20260807 18:14:20.501355 17848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 21704: Not authorized: Client connection negotiation failed: client connection to 127.17.110.2:36045: server requires authentication, but client does not have Kerberos credentials available
I20260807 18:14:20.501560 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 21704
I20260807 18:14:20.525243 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 21881
W20260807 18:14:20.528867 17848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 21881: Not authorized: Client connection negotiation failed: client connection to 127.17.110.3:45213: server requires authentication, but client does not have Kerberos credentials available
I20260807 18:14:20.529073 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 21881
I20260807 18:14:20.554945 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 21386
I20260807 18:14:20.556419 21474 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:14:20.668083 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 21386
2026-08-07T18:14:20Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4250 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:14:20 dist-test-slave-39l8 krb5kdc[22075](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:14:20 dist-test-slave-39l8 krb5kdc[22075](info): set up 2 sockets
Aug 07 18:14:20 dist-test-slave-39l8 krb5kdc[22075](info): commencing operation
krb5kdc: starting...
W20260807 18:14:22.033877 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.311s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:14:22 dist-test-slave-39l8 krb5kdc[22075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126462, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:14:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:14:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.110.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
I20260807 18:14:22.284624 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:45093
--webserver_interface=127.17.110.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:33663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:45093
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:14:22.638062 22093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:22.638458 22093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:22.638617 22093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:22.646901 22093 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:14:22.647043 22093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:22.647157 22093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:22.647274 22093 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:14:22.647367 22093 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:14:22.658162 22093 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:33663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:45093
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:45093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:22.659642 22093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:22.661054 22093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:22.674463 22099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:22.674489 22098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:22.674530 22101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:22.676012 22093 server_base.cc:1034] running on GCE node
I20260807 18:14:22.677076 22093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:22.678890 22093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:22.680153 22093 hybrid_clock.cc:648] HybridClock initialized: now 1786126462680111 us; error 63 us; skew 500 ppm
I20260807 18:14:22.680559 22093 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:22 dist-test-slave-39l8 krb5kdc[22075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126462, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:22.689229 22093 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:14:22.691131 22093 webserver.cc:460] Webserver started at http://127.17.110.62:40983/ using document root <none> and password file <none>
I20260807 18:14:22.691919 22093 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:22.692059 22093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:22.692356 22093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:22.695681 22093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "dba171564c8d480aa528b1d53058f762"
format_stamp: "Formatted at 2026-08-07 18:14:22 on dist-test-slave-39l8"
server_key: "3011dad2b3626e9214a5c9980449ae94"
server_key_iv: "99d2a2938a73446cc17bb1f1456d07e7"
server_key_version: "encryptionkey@0"
I20260807 18:14:22.696494 22093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "dba171564c8d480aa528b1d53058f762"
format_stamp: "Formatted at 2026-08-07 18:14:22 on dist-test-slave-39l8"
server_key: "3011dad2b3626e9214a5c9980449ae94"
server_key_iv: "99d2a2938a73446cc17bb1f1456d07e7"
server_key_version: "encryptionkey@0"
I20260807 18:14:22.702514 22093 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.003s
I20260807 18:14:22.706683 22111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:22.708676 22093 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:14:22.708887 22093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "dba171564c8d480aa528b1d53058f762"
format_stamp: "Formatted at 2026-08-07 18:14:22 on dist-test-slave-39l8"
server_key: "3011dad2b3626e9214a5c9980449ae94"
server_key_iv: "99d2a2938a73446cc17bb1f1456d07e7"
server_key_version: "encryptionkey@0"
I20260807 18:14:22.709095 22093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:22.733980 22093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:22.738909 22093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:22.739182 22093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:22.759951 22093 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:45093
I20260807 18:14:22.759974 22181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:45093 every 8 connection(s)
I20260807 18:14:22.761480 22093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:14:22.765658 22182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:22.769454 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 22093
I20260807 18:14:22.769726 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:14:22.770602 17848 external_mini_cluster.cc:1428] Setting key 1a3bf0f8994844b83e8fe3b22e6384be
I20260807 18:14:22.780840 22182 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dba171564c8d480aa528b1d53058f762: Bootstrap starting.
I20260807 18:14:22.784504 22182 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dba171564c8d480aa528b1d53058f762: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:22.785671 22182 log.cc:824] T 00000000000000000000000000000000 P dba171564c8d480aa528b1d53058f762: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:14:22 dist-test-slave-39l8 krb5kdc[22075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126462, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:22.788973 22182 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dba171564c8d480aa528b1d53058f762: No bootstrap required, opened a new log
I20260807 18:14:22.797734 22189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:22.773002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45804 (local address 127.17.110.62:45093)
0807 18:14:22.774015 (+ 1013us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:22.774029 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:22.774078 (+ 49us) server_negotiation.cc:407] Connection header received
0807 18:14:22.776042 (+ 1964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:22.776076 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:22.776594 (+ 518us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:22.776997 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:22.779214 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:22.780838 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:22.782463 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:22.783385 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:22.783652 (+ 267us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:22.788468 (+ 4816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:22.788478 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:22.788495 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:22.788538 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:22.792370 (+ 3832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:22.793311 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:22.793320 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:22.793329 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:22.793559 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:22.794048 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:22.794056 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:22.794068 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:22.794887 (+ 819us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:22.795286 (+ 399us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:22.795877 (+ 591us) server_negotiation.cc:299] Negotiation successful
0807 18:14:22.796473 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":755,"thread_start_us":502,"threads_started":1}
I20260807 18:14:22.797854 22182 raft_consensus.cc:348] T 00000000000000000000000000000000 P dba171564c8d480aa528b1d53058f762 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dba171564c8d480aa528b1d53058f762" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45093 } }
I20260807 18:14:22.798416 22182 raft_consensus.cc:374] T 00000000000000000000000000000000 P dba171564c8d480aa528b1d53058f762 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:22.798547 22182 raft_consensus.cc:729] T 00000000000000000000000000000000 P dba171564c8d480aa528b1d53058f762 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dba171564c8d480aa528b1d53058f762, State: Initialized, Role: FOLLOWER
I20260807 18:14:22.799321 22182 consensus_queue.cc:260] T 00000000000000000000000000000000 P dba171564c8d480aa528b1d53058f762 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dba171564c8d480aa528b1d53058f762" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45093 } }
I20260807 18:14:22.799655 22182 raft_consensus.cc:388] T 00000000000000000000000000000000 P dba171564c8d480aa528b1d53058f762 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:14:22.799818 22182 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba171564c8d480aa528b1d53058f762 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:14:22.800010 22182 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dba171564c8d480aa528b1d53058f762 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:22.802759 17848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:14:22.802999 22182 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba171564c8d480aa528b1d53058f762 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dba171564c8d480aa528b1d53058f762" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45093 } }
I20260807 18:14:22.803642 22182 leader_election.cc:302] T 00000000000000000000000000000000 P dba171564c8d480aa528b1d53058f762 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dba171564c8d480aa528b1d53058f762; no voters:
I20260807 18:14:22.804425 22182 leader_election.cc:288] T 00000000000000000000000000000000 P dba171564c8d480aa528b1d53058f762 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:14:22.804598 22191 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dba171564c8d480aa528b1d53058f762 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:14:22.807287 22182 sys_catalog.cc:564] T 00000000000000000000000000000000 P dba171564c8d480aa528b1d53058f762 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:14:22.812976 22191 raft_consensus.cc:686] T 00000000000000000000000000000000 P dba171564c8d480aa528b1d53058f762 [term 1 LEADER]: Becoming Leader. State: Replica: dba171564c8d480aa528b1d53058f762, State: Running, Role: LEADER
I20260807 18:14:22.813884 22191 consensus_queue.cc:237] T 00000000000000000000000000000000 P dba171564c8d480aa528b1d53058f762 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dba171564c8d480aa528b1d53058f762" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45093 } }
I20260807 18:14:22.820384 22192 sys_catalog.cc:455] T 00000000000000000000000000000000 P dba171564c8d480aa528b1d53058f762 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dba171564c8d480aa528b1d53058f762" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dba171564c8d480aa528b1d53058f762" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45093 } } }
I20260807 18:14:22.820919 22192 sys_catalog.cc:458] T 00000000000000000000000000000000 P dba171564c8d480aa528b1d53058f762 [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:22.821521 22193 sys_catalog.cc:455] T 00000000000000000000000000000000 P dba171564c8d480aa528b1d53058f762 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dba171564c8d480aa528b1d53058f762. Latest consensus state: current_term: 1 leader_uuid: "dba171564c8d480aa528b1d53058f762" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dba171564c8d480aa528b1d53058f762" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45093 } } }
I20260807 18:14:22.821903 22193 sys_catalog.cc:458] T 00000000000000000000000000000000 P dba171564c8d480aa528b1d53058f762 [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:22.832468 22205 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:14:22.837630 22205 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:14:22.847705 22205 catalog_manager.cc:1347] Generated new cluster ID: bb0a066d8de344d3b9b032f546a4ea18
I20260807 18:14:22.847856 22205 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:14:22.858327 22205 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:14:22.860445 22205 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:14:22.869817 22189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:22.861359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45812 (local address 127.17.110.62:45093)
0807 18:14:22.861602 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:22.861612 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:22.862273 (+ 661us) server_negotiation.cc:407] Connection header received
0807 18:14:22.862515 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:22.862520 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:22.862601 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:22.862762 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:22.863923 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:22.864765 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:22.866200 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:22.866631 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:22.866815 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:22.868179 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:22.868186 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:22.868190 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:14:22.868902 (+ 712us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:22.868973 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:14:22.869004 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:22.869078 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:22.869314 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 18:14:22.869425 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 18:14:22.877897 22205 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dba171564c8d480aa528b1d53058f762: Generated new TSK 0
I20260807 18:14:22.879076 22205 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:14:22.892331 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 22093
I20260807 18:14:22.894282 22175 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:14:22.986526 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 22093
2026-08-07T18:14:23Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2310 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:14:23 dist-test-slave-39l8 krb5kdc[22229](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:14:23 dist-test-slave-39l8 krb5kdc[22229](info): set up 2 sockets
Aug 07 18:14:23 dist-test-slave-39l8 krb5kdc[22229](info): commencing operation
krb5kdc: starting...
W20260807 18:14:24.351169 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.318s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:14:24 dist-test-slave-39l8 krb5kdc[22229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126464, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:14:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:14:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.110.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
I20260807 18:14:24.604228 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:32819
--webserver_interface=127.17.110.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:43597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:32819
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:14:24.966470 22247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:24.966936 22247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:24.967134 22247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:24.974913 22247 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:14:24.975057 22247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:24.975104 22247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:24.975211 22247 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:14:24.975286 22247 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:14:24.986373 22247 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:43597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:32819
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:32819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:24.987936 22247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:24.989341 22247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:25.002974 22253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:25.002908 22254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:25.002980 22259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:25.004416 22247 server_base.cc:1034] running on GCE node
I20260807 18:14:25.005456 22247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:25.007382 22247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:25.008690 22247 hybrid_clock.cc:648] HybridClock initialized: now 1786126465008651 us; error 61 us; skew 500 ppm
I20260807 18:14:25.009133 22247 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:25 dist-test-slave-39l8 krb5kdc[22229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126465, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:25.013373 22247 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:14:25.015393 22247 webserver.cc:460] Webserver started at http://127.17.110.62:46289/ using document root <none> and password file <none>
I20260807 18:14:25.016208 22247 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:25.016376 22247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:25.016731 22247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:25.020252 22247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "40b532e531714610987c1593aa24b91c"
format_stamp: "Formatted at 2026-08-07 18:14:25 on dist-test-slave-39l8"
server_key: "02662719fabea66570f64f35006bd7b1"
server_key_iv: "daee7f5eba1e041e55e67b73a0629bb0"
server_key_version: "encryptionkey@0"
I20260807 18:14:25.021142 22247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "40b532e531714610987c1593aa24b91c"
format_stamp: "Formatted at 2026-08-07 18:14:25 on dist-test-slave-39l8"
server_key: "02662719fabea66570f64f35006bd7b1"
server_key_iv: "daee7f5eba1e041e55e67b73a0629bb0"
server_key_version: "encryptionkey@0"
I20260807 18:14:25.027773 22247 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 18:14:25.031976 22268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:25.033794 22247 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260807 18:14:25.033982 22247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "40b532e531714610987c1593aa24b91c"
format_stamp: "Formatted at 2026-08-07 18:14:25 on dist-test-slave-39l8"
server_key: "02662719fabea66570f64f35006bd7b1"
server_key_iv: "daee7f5eba1e041e55e67b73a0629bb0"
server_key_version: "encryptionkey@0"
I20260807 18:14:25.034183 22247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:25.062335 22247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:25.067579 22247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:25.067873 22247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:25.088842 22247 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:32819
I20260807 18:14:25.088886 22342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:32819 every 8 connection(s)
I20260807 18:14:25.090412 22247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:14:25.093997 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 22247
I20260807 18:14:25.094259 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:14:25.095160 17848 external_mini_cluster.cc:1428] Setting key 284c0d33d0948c4f5adc651f2a41fd9b
I20260807 18:14:25.095288 22345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:25.111863 22345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40b532e531714610987c1593aa24b91c: Bootstrap starting.
Aug 07 18:14:25 dist-test-slave-39l8 krb5kdc[22229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126464, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:25.115517 22345 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40b532e531714610987c1593aa24b91c: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:25.116753 22345 log.cc:824] T 00000000000000000000000000000000 P 40b532e531714610987c1593aa24b91c: Log is configured to *not* fsync() on all Append() calls
I20260807 18:14:25.120887 22345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40b532e531714610987c1593aa24b91c: No bootstrap required, opened a new log
I20260807 18:14:25.123145 22348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:25.097524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56210 (local address 127.17.110.62:32819)
0807 18:14:25.098451 (+ 927us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:25.098465 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:25.098515 (+ 50us) server_negotiation.cc:407] Connection header received
0807 18:14:25.100498 (+ 1983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:25.100528 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:25.101033 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:25.101445 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:25.103667 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:25.105332 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:25.107425 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:25.108313 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:25.108489 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:25.114029 (+ 5540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:25.114038 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:25.114057 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:25.114108 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:25.117873 (+ 3765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:25.118831 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:25.118839 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:25.118848 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:25.119047 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:25.119510 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:25.119514 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:25.119516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:25.120125 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:25.120517 (+ 392us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:25.121245 (+ 728us) server_negotiation.cc:299] Negotiation successful
0807 18:14:25.121816 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":686,"thread_start_us":415,"threads_started":1}
I20260807 18:14:25.130447 22345 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40b532e531714610987c1593aa24b91c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40b532e531714610987c1593aa24b91c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 32819 } }
I20260807 18:14:25.131079 22345 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40b532e531714610987c1593aa24b91c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:25.131225 22345 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40b532e531714610987c1593aa24b91c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40b532e531714610987c1593aa24b91c, State: Initialized, Role: FOLLOWER
I20260807 18:14:25.132046 22345 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40b532e531714610987c1593aa24b91c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40b532e531714610987c1593aa24b91c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 32819 } }
I20260807 18:14:25.132457 22345 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40b532e531714610987c1593aa24b91c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:14:25.132651 22345 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40b532e531714610987c1593aa24b91c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:14:25.132874 22345 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40b532e531714610987c1593aa24b91c [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:25.137056 22345 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40b532e531714610987c1593aa24b91c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40b532e531714610987c1593aa24b91c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 32819 } }
I20260807 18:14:25.137825 22345 leader_election.cc:302] T 00000000000000000000000000000000 P 40b532e531714610987c1593aa24b91c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40b532e531714610987c1593aa24b91c; no voters:
I20260807 18:14:25.138695 22345 leader_election.cc:288] T 00000000000000000000000000000000 P 40b532e531714610987c1593aa24b91c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:14:25.138942 22351 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40b532e531714610987c1593aa24b91c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:14:25.140396 22351 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40b532e531714610987c1593aa24b91c [term 1 LEADER]: Becoming Leader. State: Replica: 40b532e531714610987c1593aa24b91c, State: Running, Role: LEADER
I20260807 18:14:25.141064 22351 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40b532e531714610987c1593aa24b91c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40b532e531714610987c1593aa24b91c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 32819 } }
I20260807 18:14:25.141876 22345 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40b532e531714610987c1593aa24b91c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:14:25.144654 22352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40b532e531714610987c1593aa24b91c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40b532e531714610987c1593aa24b91c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40b532e531714610987c1593aa24b91c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 32819 } } }
I20260807 18:14:25.145095 22352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40b532e531714610987c1593aa24b91c [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:25.145473 22353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40b532e531714610987c1593aa24b91c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40b532e531714610987c1593aa24b91c. Latest consensus state: current_term: 1 leader_uuid: "40b532e531714610987c1593aa24b91c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40b532e531714610987c1593aa24b91c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 32819 } } }
I20260807 18:14:25.146036 22353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40b532e531714610987c1593aa24b91c [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:25.152189 22357 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:14:25.158908 22357 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:14:25.171793 22357 catalog_manager.cc:1347] Generated new cluster ID: 92ca05db97d443279451c3a3b1970618
I20260807 18:14:25.172031 22357 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:14:25.186075 22357 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:14:25.187898 22357 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:14:25.195115 22357 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40b532e531714610987c1593aa24b91c: Generated new TSK 0
I20260807 18:14:25.196383 22357 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.110.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
I20260807 18:14:25.263350 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.1:0
--local_ip_for_outbound_sockets=127.17.110.1
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:32819
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:43597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:14:25.658917 22378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:25.659263 22378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:25.659365 22378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:25.667862 22378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:25.668021 22378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:25.668148 22378 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.1
I20260807 18:14:25.679309 22378 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:43597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:32819
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.1
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:25.681177 22378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:25.682646 22378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:25.700811 22388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:25.700836 22385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:25.700721 22384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:25.707211 22378 server_base.cc:1034] running on GCE node
I20260807 18:14:25.708199 22378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:25.709862 22378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:25.711477 22378 hybrid_clock.cc:648] HybridClock initialized: now 1786126465711454 us; error 218 us; skew 500 ppm
I20260807 18:14:25.711903 22378 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:25 dist-test-slave-39l8 krb5kdc[22229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126465, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:25.716094 22378 init.cc:377] Logged in from keytab as kudu/127.17.110.1@KRBTEST.COM (short username kudu)
I20260807 18:14:25.718055 22378 webserver.cc:460] Webserver started at http://127.17.110.1:41667/ using document root <none> and password file <none>
I20260807 18:14:25.718993 22378 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:25.719158 22378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:25.719506 22378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:25.722632 22378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/data/instance:
uuid: "8188c74154534f2ab678bbe066108982"
format_stamp: "Formatted at 2026-08-07 18:14:25 on dist-test-slave-39l8"
server_key: "09309baaea69f6419909af17917c1654"
server_key_iv: "a1279e4fce951c5f714cd2b979258969"
server_key_version: "encryptionkey@0"
I20260807 18:14:25.723585 22378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance:
uuid: "8188c74154534f2ab678bbe066108982"
format_stamp: "Formatted at 2026-08-07 18:14:25 on dist-test-slave-39l8"
server_key: "09309baaea69f6419909af17917c1654"
server_key_iv: "a1279e4fce951c5f714cd2b979258969"
server_key_version: "encryptionkey@0"
I20260807 18:14:25.729489 22378 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 18:14:25.733569 22399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:25.735284 22378 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 18:14:25.735487 22378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/wal
uuid: "8188c74154534f2ab678bbe066108982"
format_stamp: "Formatted at 2026-08-07 18:14:25 on dist-test-slave-39l8"
server_key: "09309baaea69f6419909af17917c1654"
server_key_iv: "a1279e4fce951c5f714cd2b979258969"
server_key_version: "encryptionkey@0"
I20260807 18:14:25.735741 22378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:25.770421 22378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:25.775137 22378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:25.775490 22378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:25.776608 22378 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:14:25.778193 22378 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:14:25.778309 22378 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:25.778472 22378 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:14:25.778554 22378 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:25.818213 22378 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.1:36287
I20260807 18:14:25.818228 22541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.1:36287 every 8 connection(s)
I20260807 18:14:25.819800 22378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
I20260807 18:14:25.822295 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 22378
I20260807 18:14:25.822556 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance
I20260807 18:14:25.823520 17848 external_mini_cluster.cc:1428] Setting key 231ab180c043dc6bb323853dbb563c7e
Aug 07 18:14:25 dist-test-slave-39l8 krb5kdc[22229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126465, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:25.851950 22545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:25.823349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:42427 (local address 127.17.110.62:32819)
0807 18:14:25.824248 (+ 899us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:25.824254 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:25.825624 (+ 1370us) server_negotiation.cc:407] Connection header received
0807 18:14:25.828745 (+ 3121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:25.828751 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:25.828837 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:25.828954 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:25.831689 (+ 2735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:25.832864 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:25.836274 (+ 3410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:25.837065 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:25.837289 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:25.842596 (+ 5307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:25.842609 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:25.842618 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:25.842684 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:25.847046 (+ 4362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:25.848464 (+ 1418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:25.848473 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:25.848480 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:25.848697 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:25.849303 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:25.849309 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:25.849313 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:25.849764 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:25.849946 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:25.851206 (+ 1260us) server_negotiation.cc:299] Negotiation successful
0807 18:14:25.851529 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":775,"thread_start_us":619,"threads_started":1}
WARNING: no policy specified for kudu/127.17.110.2@KRBTEST.COM; defaulting to no policy
I20260807 18:14:25.853130 22546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:25.824350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:32819 (local address 127.17.110.1:42427)
0807 18:14:25.825325 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:25.825392 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:25.828227 (+ 2835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:25.829330 (+ 1103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:25.829346 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:25.830325 (+ 979us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:25.831382 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:25.831422 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:25.833124 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:25.833133 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:25.835924 (+ 2791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:25.835983 (+ 59us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:25.837530 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:25.837539 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:25.837848 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:25.839082 (+ 1234us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:25.839120 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:25.842021 (+ 2901us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:25.847393 (+ 5372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:25.847409 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:25.847437 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:25.848241 (+ 804us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:25.848885 (+ 644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:25.848890 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:25.848892 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:25.849143 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:25.849988 (+ 845us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:25.849997 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:25.850514 (+ 517us) client_negotiation.cc:769] Sending connection context
0807 18:14:25.850990 (+ 476us) client_negotiation.cc:240] Negotiation successful
0807 18:14:25.851518 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":411,"threads_started":1}
I20260807 18:14:25.855482 22543 heartbeater.cc:347] Connected to a master server at 127.17.110.62:32819
I20260807 18:14:25.856227 22543 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.2@KRBTEST.COM" created.
I20260807 18:14:25.857332 22543 heartbeater.cc:511] Master 127.17.110.62:32819 requested a full tablet report, sending...
I20260807 18:14:25.860536 22291 ts_manager.cc:194] Registered new tserver with Master: 8188c74154534f2ab678bbe066108982 (127.17.110.1:36287)
I20260807 18:14:25.862217 22291 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.1@KRBTEST.COM'} at 127.17.110.1:42427
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
I20260807 18:14:25.886920 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.2:0
--local_ip_for_outbound_sockets=127.17.110.2
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:32819
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:43597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:14:26.286550 22550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:26.287133 22550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:26.287335 22550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:26.295526 22550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:26.295693 22550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:26.295888 22550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.2
I20260807 18:14:26.306341 22550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:43597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:32819
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.2
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:26.307864 22550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:26.309341 22550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:26.325104 22557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:26.325101 22558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:26.325021 22560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:26.326401 22550 server_base.cc:1034] running on GCE node
I20260807 18:14:26.327252 22550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:26.328430 22550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:26.329670 22550 hybrid_clock.cc:648] HybridClock initialized: now 1786126466329620 us; error 44 us; skew 500 ppm
I20260807 18:14:26.330068 22550 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:26 dist-test-slave-39l8 krb5kdc[22229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126466, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:26.334051 22550 init.cc:377] Logged in from keytab as kudu/127.17.110.2@KRBTEST.COM (short username kudu)
I20260807 18:14:26.336058 22550 webserver.cc:460] Webserver started at http://127.17.110.2:42493/ using document root <none> and password file <none>
I20260807 18:14:26.336930 22550 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:26.337119 22550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:26.337484 22550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:26.341393 22550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/data/instance:
uuid: "9db31065580e48648ab1828f1904ca48"
format_stamp: "Formatted at 2026-08-07 18:14:26 on dist-test-slave-39l8"
server_key: "25b1a5f83ba5d84354a6379670c898f3"
server_key_iv: "4c74e306b00f9312536c016997ab7ab0"
server_key_version: "encryptionkey@0"
I20260807 18:14:26.342610 22550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance:
uuid: "9db31065580e48648ab1828f1904ca48"
format_stamp: "Formatted at 2026-08-07 18:14:26 on dist-test-slave-39l8"
server_key: "25b1a5f83ba5d84354a6379670c898f3"
server_key_iv: "4c74e306b00f9312536c016997ab7ab0"
server_key_version: "encryptionkey@0"
I20260807 18:14:26.349363 22550 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 18:14:26.355913 22567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:26.357883 22550 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:14:26.358232 22550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/wal
uuid: "9db31065580e48648ab1828f1904ca48"
format_stamp: "Formatted at 2026-08-07 18:14:26 on dist-test-slave-39l8"
server_key: "25b1a5f83ba5d84354a6379670c898f3"
server_key_iv: "4c74e306b00f9312536c016997ab7ab0"
server_key_version: "encryptionkey@0"
I20260807 18:14:26.358487 22550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:26.388569 22550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:26.395167 22550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:26.395550 22550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:26.396827 22550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:14:26.398473 22550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:14:26.398558 22550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:26.398692 22550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:14:26.398788 22550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260807 18:14:26.442859 22550 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.2:44849
I20260807 18:14:26.442945 22715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.2:44849 every 8 connection(s)
I20260807 18:14:26.444325 22550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
I20260807 18:14:26.445231 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 22550
I20260807 18:14:26.445463 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance
I20260807 18:14:26.446024 17848 external_mini_cluster.cc:1428] Setting key 0f9b8fd2118ff2697e8c1dbc5ae2b2d9
Aug 07 18:14:26 dist-test-slave-39l8 krb5kdc[22229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126466, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.110.3@KRBTEST.COM; defaulting to no policy
I20260807 18:14:26.475452 22718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:26.447351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:58243 (local address 127.17.110.62:32819)
0807 18:14:26.447858 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:26.447865 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:26.449221 (+ 1356us) server_negotiation.cc:407] Connection header received
0807 18:14:26.452645 (+ 3424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:26.452649 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:26.452734 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:26.452840 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:26.455534 (+ 2694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:26.456384 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:26.458342 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:26.459190 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:26.459356 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:26.464616 (+ 5260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:26.464628 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:26.464637 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:26.464698 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:26.469645 (+ 4947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:26.471323 (+ 1678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:26.471338 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:26.471344 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:26.471558 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:26.472147 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:26.472155 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:26.472160 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:26.472654 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:26.472856 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:26.474871 (+ 2015us) server_negotiation.cc:299] Negotiation successful
0807 18:14:26.475106 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":271,"threads_started":1}
I20260807 18:14:26.476477 22719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:26.448043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:32819 (local address 127.17.110.2:58243)
0807 18:14:26.448996 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:26.449063 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:26.452073 (+ 3010us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:26.453206 (+ 1133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:26.453223 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:26.454240 (+ 1017us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:26.455263 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:26.455296 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:26.456599 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:26.456608 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:26.458144 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:26.458160 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:26.459448 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:26.459453 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:26.459693 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:26.460883 (+ 1190us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:26.460911 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:26.464251 (+ 3340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:26.470191 (+ 5940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:26.470219 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:26.470237 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:26.471095 (+ 858us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:26.471714 (+ 619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:26.471719 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:26.471723 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:26.471970 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:26.472998 (+ 1028us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:26.473011 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:26.474041 (+ 1030us) client_negotiation.cc:769] Sending connection context
0807 18:14:26.474368 (+ 327us) client_negotiation.cc:240] Negotiation successful
0807 18:14:26.475053 (+ 685us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":354,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.3@KRBTEST.COM" created.
I20260807 18:14:26.478816 22716 heartbeater.cc:347] Connected to a master server at 127.17.110.62:32819
I20260807 18:14:26.479209 22716 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:26.480105 22716 heartbeater.cc:511] Master 127.17.110.62:32819 requested a full tablet report, sending...
I20260807 18:14:26.481994 22291 ts_manager.cc:194] Registered new tserver with Master: 9db31065580e48648ab1828f1904ca48 (127.17.110.2:44849)
I20260807 18:14:26.483028 22291 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.2@KRBTEST.COM'} at 127.17.110.2:58243
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
I20260807 18:14:26.510926 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.3:0
--local_ip_for_outbound_sockets=127.17.110.3
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:32819
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:43597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:14:26.866197 22543 heartbeater.cc:503] Master 127.17.110.62:32819 was elected leader, sending a full tablet report...
W20260807 18:14:26.889919 22724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:26.890275 22724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:26.890457 22724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:26.898156 22724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:26.898304 22724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:26.898499 22724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.3
I20260807 18:14:26.910277 22724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:43597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:32819
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.3
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:26.911899 22724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:26.913430 22724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:26.928601 22734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:26.928678 22738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:26.928359 22735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:26.930107 22724 server_base.cc:1034] running on GCE node
I20260807 18:14:26.931022 22724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:26.932350 22724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:26.933687 22724 hybrid_clock.cc:648] HybridClock initialized: now 1786126466933593 us; error 100 us; skew 500 ppm
I20260807 18:14:26.934222 22724 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:26 dist-test-slave-39l8 krb5kdc[22229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126466, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:26.938798 22724 init.cc:377] Logged in from keytab as kudu/127.17.110.3@KRBTEST.COM (short username kudu)
I20260807 18:14:26.941052 22724 webserver.cc:460] Webserver started at http://127.17.110.3:38505/ using document root <none> and password file <none>
I20260807 18:14:26.941933 22724 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:26.942122 22724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:26.942548 22724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:26.946092 22724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/data/instance:
uuid: "9ecd33db137e47df8ae904446b302471"
format_stamp: "Formatted at 2026-08-07 18:14:26 on dist-test-slave-39l8"
server_key: "3141001909e97f360576515965317bf6"
server_key_iv: "92f41e141f7525d2f6a7ed6d362772c5"
server_key_version: "encryptionkey@0"
I20260807 18:14:26.947161 22724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance:
uuid: "9ecd33db137e47df8ae904446b302471"
format_stamp: "Formatted at 2026-08-07 18:14:26 on dist-test-slave-39l8"
server_key: "3141001909e97f360576515965317bf6"
server_key_iv: "92f41e141f7525d2f6a7ed6d362772c5"
server_key_version: "encryptionkey@0"
I20260807 18:14:26.953557 22724 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 18:14:26.958096 22747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:26.960034 22724 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:14:26.960315 22724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/wal
uuid: "9ecd33db137e47df8ae904446b302471"
format_stamp: "Formatted at 2026-08-07 18:14:26 on dist-test-slave-39l8"
server_key: "3141001909e97f360576515965317bf6"
server_key_iv: "92f41e141f7525d2f6a7ed6d362772c5"
server_key_version: "encryptionkey@0"
I20260807 18:14:26.960595 22724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:26.996701 22724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:27.003541 22724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:27.003934 22724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:27.005321 22724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:14:27.007115 22724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:14:27.007244 22724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:27.007421 22724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:14:27.007515 22724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:27.048764 22724 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.3:42543
I20260807 18:14:27.048817 22900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.3:42543 every 8 connection(s)
I20260807 18:14:27.050347 22724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
I20260807 18:14:27.060284 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 22724
I20260807 18:14:27.060525 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance
I20260807 18:14:27.061069 17848 external_mini_cluster.cc:1428] Setting key 1b6b2a3323c3551c2f5c7b734f1b51dc
Aug 07 18:14:27 dist-test-slave-39l8 krb5kdc[22229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126466, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:27.076900 22903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:27.053867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:46881 (local address 127.17.110.62:32819)
0807 18:14:27.054456 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:27.054465 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:27.055825 (+ 1360us) server_negotiation.cc:407] Connection header received
0807 18:14:27.057969 (+ 2144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:27.057974 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:27.058083 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:27.058212 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:27.060538 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:27.061599 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:27.064071 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:27.064578 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:27.064719 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:27.069286 (+ 4567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:27.069293 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:27.069297 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:27.069341 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:27.072665 (+ 3324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:27.073870 (+ 1205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:27.073876 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:27.073879 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:27.074063 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:27.074679 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:27.074685 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:27.074688 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:27.075082 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:27.075292 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:27.076220 (+ 928us) server_negotiation.cc:299] Negotiation successful
0807 18:14:27.076569 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":272,"threads_started":1}
I20260807 18:14:27.078034 22904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:27.054620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:32819 (local address 127.17.110.3:46881)
0807 18:14:27.055610 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:27.055656 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:27.057540 (+ 1884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:27.058615 (+ 1075us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:27.058627 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:27.059477 (+ 850us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:27.060277 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:27.060301 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:27.061863 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:27.061871 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:27.063831 (+ 1960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:27.063871 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:27.064893 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:27.064902 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:27.065242 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:27.066024 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:27.066051 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:27.068871 (+ 2820us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:27.072911 (+ 4040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:27.072921 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:27.072940 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:27.073660 (+ 720us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:27.074233 (+ 573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:27.074245 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:27.074248 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:27.074496 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:27.075293 (+ 797us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:27.075303 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:27.075721 (+ 418us) client_negotiation.cc:769] Sending connection context
0807 18:14:27.076057 (+ 336us) client_negotiation.cc:240] Negotiation successful
0807 18:14:27.076708 (+ 651us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"thread_start_us":422,"threads_started":1}
I20260807 18:14:27.079991 22901 heartbeater.cc:347] Connected to a master server at 127.17.110.62:32819
I20260807 18:14:27.080375 22901 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:27.081363 22901 heartbeater.cc:511] Master 127.17.110.62:32819 requested a full tablet report, sending...
I20260807 18:14:27.083519 22291 ts_manager.cc:194] Registered new tserver with Master: 9ecd33db137e47df8ae904446b302471 (127.17.110.3:42543)
I20260807 18:14:27.084640 22291 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.3@KRBTEST.COM'} at 127.17.110.3:46881
I20260807 18:14:27.091718 17848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:14:27 dist-test-slave-39l8 krb5kdc[22229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126467, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 18:14:27 dist-test-slave-39l8 krb5kdc[22229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126467, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:27.161031 22903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:27.141732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56226 (local address 127.17.110.62:32819)
0807 18:14:27.141998 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:27.142019 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:27.142904 (+ 885us) server_negotiation.cc:407] Connection header received
0807 18:14:27.143197 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:27.143204 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:27.143363 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:27.143514 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:27.145478 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:27.146323 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:27.148039 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:27.148491 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:27.148666 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:27.153755 (+ 5089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:27.153763 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:27.153768 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:27.153843 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:27.156892 (+ 3049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:27.157897 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:27.157902 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:27.157905 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:27.158380 (+ 475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:27.159150 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:27.159156 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:27.159159 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:27.159611 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:27.159876 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:27.160341 (+ 465us) server_negotiation.cc:299] Negotiation successful
0807 18:14:27.160634 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
W20260807 18:14:27.161867 22291 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:56226
I20260807 18:14:27.164778 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 22378
Aug 07 18:14:27 dist-test-slave-39l8 krb5kdc[22229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126467, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.17.110.1@KRBTEST.COM
I20260807 18:14:27.187352 22918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:27.166542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43804 (local address 127.17.110.1:36287)
0807 18:14:27.167534 (+ 992us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:27.167543 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:27.167575 (+ 32us) server_negotiation.cc:407] Connection header received
0807 18:14:27.167648 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:27.167655 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:27.167925 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:27.168088 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:27.169919 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:27.171431 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:27.172821 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:27.173502 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:27.173631 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:27.178392 (+ 4761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:27.178411 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:27.178430 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:27.178478 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:27.181906 (+ 3428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:27.183175 (+ 1269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:27.183187 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:27.183191 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:27.183432 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:27.184504 (+ 1072us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:27.184514 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:27.184518 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:27.185625 (+ 1107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:27.185942 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:27.186423 (+ 481us) server_negotiation.cc:299] Negotiation successful
0807 18:14:27.186779 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":701,"thread_start_us":483,"threads_started":1}
W20260807 18:14:27.189092 22537 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:43804
W20260807 18:14:27.190846 17848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 22378: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:14:27.191110 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 22378
I20260807 18:14:27.221306 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 22550
Aug 07 18:14:27 dist-test-slave-39l8 krb5kdc[22229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126467, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.17.110.2@KRBTEST.COM
I20260807 18:14:27.242125 22920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:27.222428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59306 (local address 127.17.110.2:44849)
0807 18:14:27.223087 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:27.223095 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:27.223123 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:14:27.223198 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:27.223204 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:27.223431 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:27.223573 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:27.225137 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:27.226375 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:27.228028 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:27.228738 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:27.228885 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:27.233868 (+ 4983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:27.233888 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:27.233906 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:27.233952 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:27.237871 (+ 3919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:27.239246 (+ 1375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:27.239256 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:27.239259 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:27.239435 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:27.239984 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:27.239988 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:27.239990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:27.240413 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:27.240681 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:27.241290 (+ 609us) server_negotiation.cc:299] Negotiation successful
0807 18:14:27.241708 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":394,"threads_started":1}
W20260807 18:14:27.243952 22710 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59306
W20260807 18:14:27.245671 17848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 22550: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:14:27.245890 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 22550
I20260807 18:14:27.277518 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 22724
Aug 07 18:14:27 dist-test-slave-39l8 krb5kdc[22229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126467, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.17.110.3@KRBTEST.COM
I20260807 18:14:27.298678 22922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:27.278809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36250 (local address 127.17.110.3:42543)
0807 18:14:27.279632 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:27.279640 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:27.279687 (+ 47us) server_negotiation.cc:407] Connection header received
0807 18:14:27.279753 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:27.279760 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:27.279981 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:27.280211 (+ 230us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:27.281699 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:27.282939 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:27.284722 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:27.285454 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:27.285651 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:27.291568 (+ 5917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:27.291592 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:27.291612 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:27.291665 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:27.295156 (+ 3491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:27.296089 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:27.296099 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:27.296101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:27.296262 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:27.296742 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:27.296747 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:27.296749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:27.297184 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:27.297497 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:27.297941 (+ 444us) server_negotiation.cc:299] Negotiation successful
0807 18:14:27.298220 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":463,"threads_started":1}
W20260807 18:14:27.300367 22894 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36250
W20260807 18:14:27.302013 17848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 22724: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:14:27.302249 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 22724
I20260807 18:14:27.333774 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 22247
I20260807 18:14:27.335417 22337 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:14:27.450919 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 22247
2026-08-07T18:14:27Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (4467 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:14:27 dist-test-slave-39l8 krb5kdc[22928](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:14:27 dist-test-slave-39l8 krb5kdc[22928](info): set up 2 sockets
Aug 07 18:14:27 dist-test-slave-39l8 krb5kdc[22928](info): commencing operation
krb5kdc: starting...
W20260807 18:14:28.786233 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.286s user 0.001s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:14:28 dist-test-slave-39l8 krb5kdc[22928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:14:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:14:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.110.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
I20260807 18:14:29.039079 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:33475
--webserver_interface=127.17.110.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:39229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:33475
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:14:29.397783 22942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:29.398139 22942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:29.398236 22942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:29.406126 22942 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:14:29.406278 22942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:29.406327 22942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:29.406378 22942 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:14:29.406425 22942 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:14:29.417040 22942 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:39229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:33475
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:33475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:29.418478 22942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:29.419926 22942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:29.432952 22949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:29.432861 22950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:29.432893 22954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:29.433861 22942 server_base.cc:1034] running on GCE node
I20260807 18:14:29.434842 22942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:29.436501 22942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:29.437798 22942 hybrid_clock.cc:648] HybridClock initialized: now 1786126469437736 us; error 79 us; skew 500 ppm
I20260807 18:14:29.438201 22942 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:29 dist-test-slave-39l8 krb5kdc[22928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126469, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:29.441809 22942 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:14:29.443553 22942 webserver.cc:460] Webserver started at http://127.17.110.62:35481/ using document root <none> and password file <none>
I20260807 18:14:29.444327 22942 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:29.444484 22942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:29.444831 22942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:29.448179 22942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "df2098a1e525447e8083a50ac3e62fea"
format_stamp: "Formatted at 2026-08-07 18:14:29 on dist-test-slave-39l8"
server_key: "b10e86dd298f5208d8a9c9c4307c0ab1"
server_key_iv: "2c08fcea52e01407a312c84062b75f42"
server_key_version: "encryptionkey@0"
I20260807 18:14:29.449055 22942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "df2098a1e525447e8083a50ac3e62fea"
format_stamp: "Formatted at 2026-08-07 18:14:29 on dist-test-slave-39l8"
server_key: "b10e86dd298f5208d8a9c9c4307c0ab1"
server_key_iv: "2c08fcea52e01407a312c84062b75f42"
server_key_version: "encryptionkey@0"
I20260807 18:14:29.455061 22942 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.003s
I20260807 18:14:29.459012 22962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:29.460588 22942 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:14:29.460834 22942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "df2098a1e525447e8083a50ac3e62fea"
format_stamp: "Formatted at 2026-08-07 18:14:29 on dist-test-slave-39l8"
server_key: "b10e86dd298f5208d8a9c9c4307c0ab1"
server_key_iv: "2c08fcea52e01407a312c84062b75f42"
server_key_version: "encryptionkey@0"
I20260807 18:14:29.461087 22942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:29.481197 22942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:29.485975 22942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:29.486310 22942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:29.506588 22942 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:33475
I20260807 18:14:29.506614 23029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:33475 every 8 connection(s)
I20260807 18:14:29.508179 22942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:14:29.512562 23030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:29.515203 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 22942
I20260807 18:14:29.515452 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:14:29.516111 17848 external_mini_cluster.cc:1428] Setting key 9b24acf703a57822f283e3ee1a56209b
I20260807 18:14:29.528162 23030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df2098a1e525447e8083a50ac3e62fea: Bootstrap starting.
I20260807 18:14:29.532290 23030 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df2098a1e525447e8083a50ac3e62fea: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:29.533455 23030 log.cc:824] T 00000000000000000000000000000000 P df2098a1e525447e8083a50ac3e62fea: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:14:29 dist-test-slave-39l8 krb5kdc[22928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:29.536896 23030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df2098a1e525447e8083a50ac3e62fea: No bootstrap required, opened a new log
I20260807 18:14:29.545382 23034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:29.518539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35040 (local address 127.17.110.62:33475)
0807 18:14:29.519672 (+ 1133us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:29.519694 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:29.519756 (+ 62us) server_negotiation.cc:407] Connection header received
0807 18:14:29.522474 (+ 2718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:29.522511 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:29.523152 (+ 641us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:29.523714 (+ 562us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:29.525378 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:29.527697 (+ 2319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:29.529268 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:29.530549 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:29.530828 (+ 279us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:29.535777 (+ 4949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:29.535786 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:29.535805 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:29.535847 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:29.539711 (+ 3864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:29.540578 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:29.540586 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:29.540594 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:29.540798 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:29.541390 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:29.541397 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:29.541401 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:29.542313 (+ 912us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:29.542667 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:29.543177 (+ 510us) server_negotiation.cc:299] Negotiation successful
0807 18:14:29.543717 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":827,"thread_start_us":469,"threads_started":1}
I20260807 18:14:29.547806 23030 raft_consensus.cc:348] T 00000000000000000000000000000000 P df2098a1e525447e8083a50ac3e62fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df2098a1e525447e8083a50ac3e62fea" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 33475 } }
I20260807 18:14:29.548244 23030 raft_consensus.cc:374] T 00000000000000000000000000000000 P df2098a1e525447e8083a50ac3e62fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:29.548403 23030 raft_consensus.cc:729] T 00000000000000000000000000000000 P df2098a1e525447e8083a50ac3e62fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df2098a1e525447e8083a50ac3e62fea, State: Initialized, Role: FOLLOWER
I20260807 18:14:29.549163 23030 consensus_queue.cc:260] T 00000000000000000000000000000000 P df2098a1e525447e8083a50ac3e62fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df2098a1e525447e8083a50ac3e62fea" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 33475 } }
I20260807 18:14:29.549458 23030 raft_consensus.cc:388] T 00000000000000000000000000000000 P df2098a1e525447e8083a50ac3e62fea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:14:29.549618 23030 raft_consensus.cc:482] T 00000000000000000000000000000000 P df2098a1e525447e8083a50ac3e62fea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:14:29.549813 23030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df2098a1e525447e8083a50ac3e62fea [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:29.552686 23030 raft_consensus.cc:504] T 00000000000000000000000000000000 P df2098a1e525447e8083a50ac3e62fea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df2098a1e525447e8083a50ac3e62fea" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 33475 } }
I20260807 18:14:29.553375 23030 leader_election.cc:302] T 00000000000000000000000000000000 P df2098a1e525447e8083a50ac3e62fea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df2098a1e525447e8083a50ac3e62fea; no voters:
I20260807 18:14:29.554134 23030 leader_election.cc:288] T 00000000000000000000000000000000 P df2098a1e525447e8083a50ac3e62fea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:14:29.554342 23037 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df2098a1e525447e8083a50ac3e62fea [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:14:29.555581 23037 raft_consensus.cc:686] T 00000000000000000000000000000000 P df2098a1e525447e8083a50ac3e62fea [term 1 LEADER]: Becoming Leader. State: Replica: df2098a1e525447e8083a50ac3e62fea, State: Running, Role: LEADER
I20260807 18:14:29.556263 23037 consensus_queue.cc:237] T 00000000000000000000000000000000 P df2098a1e525447e8083a50ac3e62fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df2098a1e525447e8083a50ac3e62fea" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 33475 } }
I20260807 18:14:29.556615 23030 sys_catalog.cc:564] T 00000000000000000000000000000000 P df2098a1e525447e8083a50ac3e62fea [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:14:29.559564 23040 sys_catalog.cc:455] T 00000000000000000000000000000000 P df2098a1e525447e8083a50ac3e62fea [sys.catalog]: SysCatalogTable state changed. Reason: New leader df2098a1e525447e8083a50ac3e62fea. Latest consensus state: current_term: 1 leader_uuid: "df2098a1e525447e8083a50ac3e62fea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df2098a1e525447e8083a50ac3e62fea" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 33475 } } }
I20260807 18:14:29.559996 23040 sys_catalog.cc:458] T 00000000000000000000000000000000 P df2098a1e525447e8083a50ac3e62fea [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:29.561090 23039 sys_catalog.cc:455] T 00000000000000000000000000000000 P df2098a1e525447e8083a50ac3e62fea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df2098a1e525447e8083a50ac3e62fea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df2098a1e525447e8083a50ac3e62fea" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 33475 } } }
I20260807 18:14:29.561476 23039 sys_catalog.cc:458] T 00000000000000000000000000000000 P df2098a1e525447e8083a50ac3e62fea [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:29.569233 23046 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:14:29.575124 23046 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:14:29.585879 23046 catalog_manager.cc:1347] Generated new cluster ID: cfc36e0da3c643aeae03435fc7894c8f
I20260807 18:14:29.586040 23046 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:14:29.608494 23046 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:14:29.609980 23046 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:14:29.619293 23046 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df2098a1e525447e8083a50ac3e62fea: Generated new TSK 0
I20260807 18:14:29.620656 23046 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.110.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
I20260807 18:14:29.692943 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.1:0
--local_ip_for_outbound_sockets=127.17.110.1
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:33475
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:39229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:14:30.060426 23062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:30.060740 23062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:30.060839 23062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:30.068641 23062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:30.068769 23062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:30.068889 23062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.1
I20260807 18:14:30.079276 23062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:39229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:33475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.1
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:30.080621 23062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:30.082070 23062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:30.096391 23073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:30.096364 23068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:30.096298 23071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:30.097249 23062 server_base.cc:1034] running on GCE node
I20260807 18:14:30.097939 23062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:30.099174 23062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:30.100466 23062 hybrid_clock.cc:648] HybridClock initialized: now 1786126470100384 us; error 71 us; skew 500 ppm
I20260807 18:14:30.100880 23062 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:30 dist-test-slave-39l8 krb5kdc[22928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126470, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:30.104813 23062 init.cc:377] Logged in from keytab as kudu/127.17.110.1@KRBTEST.COM (short username kudu)
I20260807 18:14:30.106873 23062 webserver.cc:460] Webserver started at http://127.17.110.1:41593/ using document root <none> and password file <none>
I20260807 18:14:30.107674 23062 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:30.107798 23062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:30.108116 23062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:30.111698 23062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-0/data/instance:
uuid: "3df427883ba54a9bae333eaa419dd86d"
format_stamp: "Formatted at 2026-08-07 18:14:30 on dist-test-slave-39l8"
server_key: "df563da8e922ed2a23ff5a2c6fda9e88"
server_key_iv: "9dc1bfdb39757390e0ab20ed4ec3aad9"
server_key_version: "encryptionkey@0"
I20260807 18:14:30.112643 23062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance:
uuid: "3df427883ba54a9bae333eaa419dd86d"
format_stamp: "Formatted at 2026-08-07 18:14:30 on dist-test-slave-39l8"
server_key: "df563da8e922ed2a23ff5a2c6fda9e88"
server_key_iv: "9dc1bfdb39757390e0ab20ed4ec3aad9"
server_key_version: "encryptionkey@0"
I20260807 18:14:30.119234 23062 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260807 18:14:30.123504 23081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:30.125133 23062 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:14:30.125362 23062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-0/wal
uuid: "3df427883ba54a9bae333eaa419dd86d"
format_stamp: "Formatted at 2026-08-07 18:14:30 on dist-test-slave-39l8"
server_key: "df563da8e922ed2a23ff5a2c6fda9e88"
server_key_iv: "9dc1bfdb39757390e0ab20ed4ec3aad9"
server_key_version: "encryptionkey@0"
I20260807 18:14:30.125597 23062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:30.140801 23062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:30.145428 23062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:30.145777 23062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:30.146998 23062 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:14:30.148512 23062 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:14:30.148593 23062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:30.148761 23062 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:14:30.148820 23062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:30.186375 23062 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.1:44175
I20260807 18:14:30.186414 23236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.1:44175 every 8 connection(s)
I20260807 18:14:30.187831 23062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
I20260807 18:14:30.188946 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 23062
I20260807 18:14:30.189236 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance
I20260807 18:14:30.190060 17848 external_mini_cluster.cc:1428] Setting key f57c1782c308c70009d5700645f0b4a2
Aug 07 18:14:30 dist-test-slave-39l8 krb5kdc[22928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126470, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:30.212687 23240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:30.191136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:46391 (local address 127.17.110.62:33475)
0807 18:14:30.191707 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:30.191714 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:30.192995 (+ 1281us) server_negotiation.cc:407] Connection header received
0807 18:14:30.195201 (+ 2206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:30.195207 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:30.195318 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:30.195457 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:30.197319 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:30.198163 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:30.200428 (+ 2265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:30.200869 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:30.200995 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:30.205908 (+ 4913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:30.205915 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:30.205919 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:30.205957 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:30.208916 (+ 2959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:30.209859 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:30.209871 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:30.209874 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:30.210010 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:30.210625 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:30.210630 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:30.210632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:30.210987 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:30.211162 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:30.212027 (+ 865us) server_negotiation.cc:299] Negotiation successful
0807 18:14:30.212306 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":288,"threads_started":1}
I20260807 18:14:30.213785 23241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:30.191782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:33475 (local address 127.17.110.1:46391)
0807 18:14:30.192810 (+ 1028us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:30.192856 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:30.194877 (+ 2021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:30.195756 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:30.195765 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:30.196380 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:30.197059 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:30.197081 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:30.198358 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:30.198363 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:30.200219 (+ 1856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:30.200243 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:30.201165 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:30.201177 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:30.201579 (+ 402us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:30.202463 (+ 884us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:30.202504 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:30.205595 (+ 3091us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:30.209130 (+ 3535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:30.209145 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:30.209162 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:30.209693 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:30.210145 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:30.210151 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:30.210154 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:30.210467 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:30.211139 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:30.211149 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:30.211561 (+ 412us) client_negotiation.cc:769] Sending connection context
0807 18:14:30.211869 (+ 308us) client_negotiation.cc:240] Negotiation successful
0807 18:14:30.212508 (+ 639us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"thread_start_us":424,"threads_started":1}
WARNING: no policy specified for kudu/127.17.110.2@KRBTEST.COM; defaulting to no policy
I20260807 18:14:30.215766 23237 heartbeater.cc:347] Connected to a master server at 127.17.110.62:33475
I20260807 18:14:30.216246 23237 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:30.217120 23237 heartbeater.cc:511] Master 127.17.110.62:33475 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.2@KRBTEST.COM" created.
I20260807 18:14:30.219696 22981 ts_manager.cc:194] Registered new tserver with Master: 3df427883ba54a9bae333eaa419dd86d (127.17.110.1:44175)
I20260807 18:14:30.221304 22981 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.1@KRBTEST.COM'} at 127.17.110.1:46391
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
I20260807 18:14:30.247040 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.2:0
--local_ip_for_outbound_sockets=127.17.110.2
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:33475
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:39229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:14:30.614482 23244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:30.614953 23244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:30.615145 23244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:30.623150 23244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:30.623332 23244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:30.623512 23244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.2
I20260807 18:14:30.633891 23244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:39229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:33475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.2
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:30.635414 23244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:30.636770 23244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:30.651880 23253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:30.651844 23252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:30.652185 23256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:30.653208 23244 server_base.cc:1034] running on GCE node
I20260807 18:14:30.654081 23244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:30.655354 23244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:30.656632 23244 hybrid_clock.cc:648] HybridClock initialized: now 1786126470656593 us; error 68 us; skew 500 ppm
I20260807 18:14:30.657025 23244 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:30 dist-test-slave-39l8 krb5kdc[22928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126470, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:30.660835 23244 init.cc:377] Logged in from keytab as kudu/127.17.110.2@KRBTEST.COM (short username kudu)
I20260807 18:14:30.662667 23244 webserver.cc:460] Webserver started at http://127.17.110.2:42131/ using document root <none> and password file <none>
I20260807 18:14:30.663556 23244 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:30.663684 23244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:30.663991 23244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:30.667785 23244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-1/data/instance:
uuid: "858b534700534c9db40e84783e0a143b"
format_stamp: "Formatted at 2026-08-07 18:14:30 on dist-test-slave-39l8"
server_key: "f179632a0a59609e318958d087ff2bd0"
server_key_iv: "81a46d72680378dbdc0460cf840fda75"
server_key_version: "encryptionkey@0"
I20260807 18:14:30.668728 23244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance:
uuid: "858b534700534c9db40e84783e0a143b"
format_stamp: "Formatted at 2026-08-07 18:14:30 on dist-test-slave-39l8"
server_key: "f179632a0a59609e318958d087ff2bd0"
server_key_iv: "81a46d72680378dbdc0460cf840fda75"
server_key_version: "encryptionkey@0"
I20260807 18:14:30.675501 23244 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 18:14:30.680286 23264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:30.682106 23244 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260807 18:14:30.682296 23244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-1/wal
uuid: "858b534700534c9db40e84783e0a143b"
format_stamp: "Formatted at 2026-08-07 18:14:30 on dist-test-slave-39l8"
server_key: "f179632a0a59609e318958d087ff2bd0"
server_key_iv: "81a46d72680378dbdc0460cf840fda75"
server_key_version: "encryptionkey@0"
I20260807 18:14:30.682516 23244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:30.714895 23244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:30.720458 23244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:30.720821 23244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:30.721978 23244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:14:30.723650 23244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:14:30.723737 23244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:30.723889 23244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:14:30.723948 23244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:30.764282 23244 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.2:37745
I20260807 18:14:30.764340 23420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.2:37745 every 8 connection(s)
I20260807 18:14:30.765973 23244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
I20260807 18:14:30.773861 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 23244
I20260807 18:14:30.774114 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance
I20260807 18:14:30.774618 17848 external_mini_cluster.cc:1428] Setting key db53490020734ab41ba372faadd501fa
Aug 07 18:14:30 dist-test-slave-39l8 krb5kdc[22928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126470, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:30.791371 23424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:30.769101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:52575 (local address 127.17.110.62:33475)
0807 18:14:30.769628 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:30.769636 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:30.770870 (+ 1234us) server_negotiation.cc:407] Connection header received
0807 18:14:30.773257 (+ 2387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:30.773262 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:30.773395 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:30.773560 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:30.775730 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:30.776734 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:30.778903 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:30.779418 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:30.779598 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:30.784825 (+ 5227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:30.784831 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:30.784835 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:30.784879 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:30.787911 (+ 3032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:30.788788 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:30.788799 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:30.788801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:30.788939 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:30.789442 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:30.789447 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:30.789449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:30.789741 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:30.789912 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:30.790687 (+ 775us) server_negotiation.cc:299] Negotiation successful
0807 18:14:30.790969 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":247,"threads_started":1}
I20260807 18:14:30.792536 23425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:30.769741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:33475 (local address 127.17.110.2:52575)
0807 18:14:30.770607 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:30.770652 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:30.772916 (+ 2264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:30.773832 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:30.773842 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:30.774586 (+ 744us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:30.775492 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:30.775529 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:30.776973 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:30.776979 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:30.778638 (+ 1659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:30.778662 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:30.779726 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:30.779734 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:30.780014 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:30.780886 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:30.780917 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:30.784453 (+ 3536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:30.788105 (+ 3652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:30.788115 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:30.788134 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:30.788634 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:30.789074 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:30.789079 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:30.789083 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:30.789291 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:30.789884 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:30.789893 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:30.790285 (+ 392us) client_negotiation.cc:769] Sending connection context
0807 18:14:30.790575 (+ 290us) client_negotiation.cc:240] Negotiation successful
0807 18:14:30.791232 (+ 657us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":311,"threads_started":1}
I20260807 18:14:30.794525 23422 heartbeater.cc:347] Connected to a master server at 127.17.110.62:33475
I20260807 18:14:30.794960 23422 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:30.795811 23422 heartbeater.cc:511] Master 127.17.110.62:33475 requested a full tablet report, sending...
I20260807 18:14:30.797678 22981 ts_manager.cc:194] Registered new tserver with Master: 858b534700534c9db40e84783e0a143b (127.17.110.2:37745)
I20260807 18:14:30.798684 22981 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.2@KRBTEST.COM'} at 127.17.110.2:52575
WARNING: no policy specified for kudu/127.17.110.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
I20260807 18:14:30.832370 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.3:0
--local_ip_for_outbound_sockets=127.17.110.3
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:33475
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:39229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:14:31.199095 23428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:31.199417 23428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:31.199515 23428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:31.207340 23428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:31.207472 23428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:31.207589 23428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.3
I20260807 18:14:31.217877 23428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:39229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:33475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.3
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:31.219259 23428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:31.220649 23428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:14:31.224745 23237 heartbeater.cc:503] Master 127.17.110.62:33475 was elected leader, sending a full tablet report...
W20260807 18:14:31.236238 23433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:31.236238 23437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:31.236526 23439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:31.237970 23428 server_base.cc:1034] running on GCE node
I20260807 18:14:31.238888 23428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:31.240236 23428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:31.241555 23428 hybrid_clock.cc:648] HybridClock initialized: now 1786126471241475 us; error 74 us; skew 500 ppm
I20260807 18:14:31.241984 23428 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:31 dist-test-slave-39l8 krb5kdc[22928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126471, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:31.246133 23428 init.cc:377] Logged in from keytab as kudu/127.17.110.3@KRBTEST.COM (short username kudu)
I20260807 18:14:31.248261 23428 webserver.cc:460] Webserver started at http://127.17.110.3:36955/ using document root <none> and password file <none>
I20260807 18:14:31.249111 23428 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:31.249305 23428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:31.249697 23428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:31.253428 23428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-2/data/instance:
uuid: "4b4f5a1f1a79465f93ab5dc8091879e7"
format_stamp: "Formatted at 2026-08-07 18:14:31 on dist-test-slave-39l8"
server_key: "82fefa1905a224175367ab6b67a2a585"
server_key_iv: "c9ac950718ffee2bd917f28295062af0"
server_key_version: "encryptionkey@0"
I20260807 18:14:31.254491 23428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance:
uuid: "4b4f5a1f1a79465f93ab5dc8091879e7"
format_stamp: "Formatted at 2026-08-07 18:14:31 on dist-test-slave-39l8"
server_key: "82fefa1905a224175367ab6b67a2a585"
server_key_iv: "c9ac950718ffee2bd917f28295062af0"
server_key_version: "encryptionkey@0"
I20260807 18:14:31.261554 23428 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 18:14:31.266525 23450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:31.268484 23428 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:14:31.268740 23428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-2/wal
uuid: "4b4f5a1f1a79465f93ab5dc8091879e7"
format_stamp: "Formatted at 2026-08-07 18:14:31 on dist-test-slave-39l8"
server_key: "82fefa1905a224175367ab6b67a2a585"
server_key_iv: "c9ac950718ffee2bd917f28295062af0"
server_key_version: "encryptionkey@0"
I20260807 18:14:31.269024 23428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:31.288326 23428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:31.293777 23428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:31.294181 23428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:31.295575 23428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:14:31.297235 23428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:14:31.297358 23428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:31.297535 23428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:14:31.297628 23428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:31.337782 23428 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.3:45399
I20260807 18:14:31.337843 23605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.3:45399 every 8 connection(s)
I20260807 18:14:31.339362 23428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
I20260807 18:14:31.349539 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 23428
I20260807 18:14:31.349753 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance
I20260807 18:14:31.350205 17848 external_mini_cluster.cc:1428] Setting key a8d4d0332f880e3d794d81414d888faf
Aug 07 18:14:31 dist-test-slave-39l8 krb5kdc[22928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126471, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:31.364517 23610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:31.342547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:44701 (local address 127.17.110.62:33475)
0807 18:14:31.343133 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:31.343138 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:31.344433 (+ 1295us) server_negotiation.cc:407] Connection header received
0807 18:14:31.346810 (+ 2377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:31.346817 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:31.346941 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:31.347085 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:31.348913 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:31.349956 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:31.352304 (+ 2348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:31.352928 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:31.353090 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:31.357861 (+ 4771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:31.357868 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:31.357872 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:31.357932 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:31.361073 (+ 3141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:31.362062 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:31.362066 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:31.362069 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:31.362214 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:31.362693 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:31.362696 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:31.362698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:31.363067 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:31.363202 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:31.363967 (+ 765us) server_negotiation.cc:299] Negotiation successful
0807 18:14:31.364162 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":328,"threads_started":1}
I20260807 18:14:31.365969 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:31.343334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:33475 (local address 127.17.110.3:44701)
0807 18:14:31.344234 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:31.344291 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:31.346404 (+ 2113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:31.347391 (+ 987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:31.347400 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:31.348010 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:31.348685 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:31.348706 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:31.350164 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:31.350168 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:31.352123 (+ 1955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:31.352142 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:31.353218 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:31.353230 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:31.353569 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:31.354459 (+ 890us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:31.354487 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:31.357546 (+ 3059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:31.361321 (+ 3775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:31.361331 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:31.361347 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:31.361917 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:31.362344 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:31.362348 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:31.362351 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:31.362580 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:31.363195 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:31.363202 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:31.363582 (+ 380us) client_negotiation.cc:769] Sending connection context
0807 18:14:31.363842 (+ 260us) client_negotiation.cc:240] Negotiation successful
0807 18:14:31.364377 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":347,"threads_started":1}
I20260807 18:14:31.367769 23606 heartbeater.cc:347] Connected to a master server at 127.17.110.62:33475
I20260807 18:14:31.368136 23606 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:31.368974 23606 heartbeater.cc:511] Master 127.17.110.62:33475 requested a full tablet report, sending...
I20260807 18:14:31.370863 22981 ts_manager.cc:194] Registered new tserver with Master: 4b4f5a1f1a79465f93ab5dc8091879e7 (127.17.110.3:45399)
I20260807 18:14:31.371882 22981 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.3@KRBTEST.COM'} at 127.17.110.3:44701
I20260807 18:14:31.380642 17848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:14:31 dist-test-slave-39l8 krb5kdc[22928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 18:14:31 dist-test-slave-39l8 krb5kdc[22928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.1@KRBTEST.COM
I20260807 18:14:31.443331 23618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:31.425235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37482 (local address 127.17.110.1:44175)
0807 18:14:31.425999 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:31.426007 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:31.426033 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:14:31.426096 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:31.426104 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:31.426322 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:31.426506 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:31.428166 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:31.429300 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:31.430777 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:31.431364 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:31.431522 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:31.436636 (+ 5114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:31.436655 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:31.436673 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:31.436728 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:31.440029 (+ 3301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:31.440854 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:31.440862 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:31.440865 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:31.441004 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:31.441453 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:31.441458 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:31.441464 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:31.441922 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:31.442268 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:31.442605 (+ 337us) server_negotiation.cc:299] Negotiation successful
0807 18:14:31.442941 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":393,"threads_started":1}
Aug 07 18:14:31 dist-test-slave-39l8 krb5kdc[22928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:31.465070 23610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:31.448664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52942 (local address 127.17.110.62:33475)
0807 18:14:31.448944 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:31.448949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:31.449107 (+ 158us) server_negotiation.cc:407] Connection header received
0807 18:14:31.449211 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:31.449215 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:31.449289 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:31.449382 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:31.450872 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:31.451674 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:31.453016 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:31.453584 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:31.453727 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:31.458429 (+ 4702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:31.458436 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:31.458440 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:31.458491 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:31.461889 (+ 3398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:31.462886 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:31.462892 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:31.462894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:31.463051 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:31.463521 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:31.463526 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:31.463528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:31.463903 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:31.464104 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:31.464474 (+ 370us) server_negotiation.cc:299] Negotiation successful
0807 18:14:31.464705 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
W20260807 18:14:31.465852 22981 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52942
I20260807 18:14:31.467106 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 23062
I20260807 18:14:31.483513 23618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:31.468289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37484 (local address 127.17.110.1:44175)
0807 18:14:31.468613 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:31.468623 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:31.468841 (+ 218us) server_negotiation.cc:407] Connection header received
0807 18:14:31.469080 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:31.469086 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:31.469187 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:31.469315 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:31.470884 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:31.471618 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:31.473408 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:31.473955 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:31.474154 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:31.477085 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:31.477091 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:31.477095 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:31.477140 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:31.480383 (+ 3243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:31.481182 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:31.481186 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:31.481189 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:31.481353 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:31.481809 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:31.481814 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:31.481816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:31.482136 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:31.482345 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:31.482812 (+ 467us) server_negotiation.cc:299] Negotiation successful
0807 18:14:31.483029 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260807 18:14:31.484408 23232 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:14:31.572196 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 23062
I20260807 18:14:31.594363 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 23244
Aug 07 18:14:31 dist-test-slave-39l8 krb5kdc[22928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.2@KRBTEST.COM
I20260807 18:14:31.612979 23627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:31.595565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42978 (local address 127.17.110.2:37745)
0807 18:14:31.596210 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:31.596218 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:31.596248 (+ 30us) server_negotiation.cc:407] Connection header received
0807 18:14:31.596316 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:31.596323 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:31.596542 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:31.596684 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:31.598066 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:31.599350 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:31.600697 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:31.601293 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:31.601411 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:31.606149 (+ 4738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:31.606167 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:31.606185 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:31.606229 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:31.609564 (+ 3335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:31.610468 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:31.610478 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:31.610480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:31.610636 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:31.611239 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:31.611242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:31.611245 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:31.611626 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:31.611897 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:31.612285 (+ 388us) server_negotiation.cc:299] Negotiation successful
0807 18:14:31.612519 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":339,"threads_started":1}
I20260807 18:14:31.614526 23415 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:14:31.701016 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 23244
I20260807 18:14:31.726846 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 23428
Aug 07 18:14:31 dist-test-slave-39l8 krb5kdc[22928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.3@KRBTEST.COM
I20260807 18:14:31.746033 23631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:31.727938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40478 (local address 127.17.110.3:45399)
0807 18:14:31.728605 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:31.728613 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:31.728662 (+ 49us) server_negotiation.cc:407] Connection header received
0807 18:14:31.728725 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:31.728732 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:31.728948 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:31.729089 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:31.730549 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:31.731770 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:31.733167 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:31.733758 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:31.733863 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:31.738704 (+ 4841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:31.738792 (+ 88us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:31.738817 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:31.738866 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:31.742427 (+ 3561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:31.743498 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:31.743508 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:31.743511 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:31.743714 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:31.744166 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:31.744172 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:31.744175 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:31.744621 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:31.744925 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:31.745273 (+ 348us) server_negotiation.cc:299] Negotiation successful
0807 18:14:31.745525 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":350,"threads_started":1}
I20260807 18:14:31.747804 23597 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:14:31.837770 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 23428
I20260807 18:14:31.865115 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 22942
I20260807 18:14:31.866627 23024 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:14:31.976605 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 22942
2026-08-07T18:14:32Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4535 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:14:32 dist-test-slave-39l8 krb5kdc[23640](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:14:32 dist-test-slave-39l8 krb5kdc[23640](info): set up 2 sockets
Aug 07 18:14:32 dist-test-slave-39l8 krb5kdc[23640](info): commencing operation
krb5kdc: starting...
W20260807 18:14:33.357836 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.322s user 0.008s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:14:33 dist-test-slave-39l8 krb5kdc[23640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:14:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:14:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.110.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
I20260807 18:14:33.637125 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:44163
--webserver_interface=127.17.110.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:41733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:44163
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:14:34.018728 23655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:34.019366 23655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:34.019591 23655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:34.030005 23655 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:14:34.030160 23655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:34.030256 23655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:34.030346 23655 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:14:34.030440 23655 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:14:34.041638 23655 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:41733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:44163
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:44163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:34.043251 23655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:34.044703 23655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:34.058769 23664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:34.058823 23667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:34.058765 23665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:34.065169 23655 server_base.cc:1034] running on GCE node
I20260807 18:14:34.066309 23655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:34.068322 23655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:34.069679 23655 hybrid_clock.cc:648] HybridClock initialized: now 1786126474069611 us; error 56 us; skew 500 ppm
I20260807 18:14:34.070230 23655 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:34 dist-test-slave-39l8 krb5kdc[23640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126474, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:34.079531 23655 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:14:34.081240 23655 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:34.081370 23655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:34.081704 23655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:34.085287 23655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "382bc085a55b4a5cbe6457119a6ef7fa"
format_stamp: "Formatted at 2026-08-07 18:14:34 on dist-test-slave-39l8"
server_key: "1d7ad10512961bbbd436432a4cc6fed8"
server_key_iv: "97e4044432ab065d3a1123f69c1edf55"
server_key_version: "encryptionkey@0"
I20260807 18:14:34.086237 23655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "382bc085a55b4a5cbe6457119a6ef7fa"
format_stamp: "Formatted at 2026-08-07 18:14:34 on dist-test-slave-39l8"
server_key: "1d7ad10512961bbbd436432a4cc6fed8"
server_key_iv: "97e4044432ab065d3a1123f69c1edf55"
server_key_version: "encryptionkey@0"
I20260807 18:14:34.093376 23655 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260807 18:14:34.098152 23676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:34.100384 23655 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:14:34.100638 23655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "382bc085a55b4a5cbe6457119a6ef7fa"
format_stamp: "Formatted at 2026-08-07 18:14:34 on dist-test-slave-39l8"
server_key: "1d7ad10512961bbbd436432a4cc6fed8"
server_key_iv: "97e4044432ab065d3a1123f69c1edf55"
server_key_version: "encryptionkey@0"
I20260807 18:14:34.100906 23655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:34.128170 23655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:34.135084 23655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:34.135381 23655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:34.159475 23655 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:44163
I20260807 18:14:34.159530 23747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:44163 every 8 connection(s)
I20260807 18:14:34.160778 23655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:14:34.165488 23748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:34.165795 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 23655
I20260807 18:14:34.166064 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:14:34.166826 17848 external_mini_cluster.cc:1428] Setting key 3750fb2f38bc3191fe1c690066ecd4f2
I20260807 18:14:34.182821 23748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 382bc085a55b4a5cbe6457119a6ef7fa: Bootstrap starting.
Aug 07 18:14:34 dist-test-slave-39l8 krb5kdc[23640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:34.186513 23748 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 382bc085a55b4a5cbe6457119a6ef7fa: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:34.188072 23748 log.cc:824] T 00000000000000000000000000000000 P 382bc085a55b4a5cbe6457119a6ef7fa: Log is configured to *not* fsync() on all Append() calls
I20260807 18:14:34.192531 23748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 382bc085a55b4a5cbe6457119a6ef7fa: No bootstrap required, opened a new log
I20260807 18:14:34.198585 23754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:34.169258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60576 (local address 127.17.110.62:44163)
0807 18:14:34.170295 (+ 1037us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:34.170317 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:34.170375 (+ 58us) server_negotiation.cc:407] Connection header received
0807 18:14:34.173179 (+ 2804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:34.173233 (+ 54us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:34.173927 (+ 694us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:34.174495 (+ 568us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:34.176225 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:34.178458 (+ 2233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:34.180988 (+ 2530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:34.182231 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:34.182518 (+ 287us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:34.187266 (+ 4748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:34.187281 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:34.187303 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:34.187360 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:34.191955 (+ 4595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:34.192923 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:34.192938 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:34.192948 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:34.193256 (+ 308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:34.193854 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:34.193860 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:34.193863 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:34.194784 (+ 921us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:34.195190 (+ 406us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:34.196030 (+ 840us) server_negotiation.cc:299] Negotiation successful
0807 18:14:34.196649 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":740,"thread_start_us":485,"threads_started":1}
I20260807 18:14:34.204413 23748 raft_consensus.cc:348] T 00000000000000000000000000000000 P 382bc085a55b4a5cbe6457119a6ef7fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "382bc085a55b4a5cbe6457119a6ef7fa" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 44163 } }
I20260807 18:14:34.204864 23748 raft_consensus.cc:374] T 00000000000000000000000000000000 P 382bc085a55b4a5cbe6457119a6ef7fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:34.205034 23748 raft_consensus.cc:729] T 00000000000000000000000000000000 P 382bc085a55b4a5cbe6457119a6ef7fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 382bc085a55b4a5cbe6457119a6ef7fa, State: Initialized, Role: FOLLOWER
I20260807 18:14:34.205798 23748 consensus_queue.cc:260] T 00000000000000000000000000000000 P 382bc085a55b4a5cbe6457119a6ef7fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "382bc085a55b4a5cbe6457119a6ef7fa" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 44163 } }
I20260807 18:14:34.206084 23748 raft_consensus.cc:388] T 00000000000000000000000000000000 P 382bc085a55b4a5cbe6457119a6ef7fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:14:34.206239 23748 raft_consensus.cc:482] T 00000000000000000000000000000000 P 382bc085a55b4a5cbe6457119a6ef7fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:14:34.206439 23748 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 382bc085a55b4a5cbe6457119a6ef7fa [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:34.209187 23748 raft_consensus.cc:504] T 00000000000000000000000000000000 P 382bc085a55b4a5cbe6457119a6ef7fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "382bc085a55b4a5cbe6457119a6ef7fa" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 44163 } }
I20260807 18:14:34.209972 23748 leader_election.cc:302] T 00000000000000000000000000000000 P 382bc085a55b4a5cbe6457119a6ef7fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 382bc085a55b4a5cbe6457119a6ef7fa; no voters:
I20260807 18:14:34.210796 23748 leader_election.cc:288] T 00000000000000000000000000000000 P 382bc085a55b4a5cbe6457119a6ef7fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:14:34.210948 23756 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 382bc085a55b4a5cbe6457119a6ef7fa [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:14:34.211755 23756 raft_consensus.cc:686] T 00000000000000000000000000000000 P 382bc085a55b4a5cbe6457119a6ef7fa [term 1 LEADER]: Becoming Leader. State: Replica: 382bc085a55b4a5cbe6457119a6ef7fa, State: Running, Role: LEADER
I20260807 18:14:34.212389 23756 consensus_queue.cc:237] T 00000000000000000000000000000000 P 382bc085a55b4a5cbe6457119a6ef7fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "382bc085a55b4a5cbe6457119a6ef7fa" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 44163 } }
I20260807 18:14:34.213285 23748 sys_catalog.cc:564] T 00000000000000000000000000000000 P 382bc085a55b4a5cbe6457119a6ef7fa [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:14:34.215767 23761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 382bc085a55b4a5cbe6457119a6ef7fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 382bc085a55b4a5cbe6457119a6ef7fa. Latest consensus state: current_term: 1 leader_uuid: "382bc085a55b4a5cbe6457119a6ef7fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "382bc085a55b4a5cbe6457119a6ef7fa" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 44163 } } }
I20260807 18:14:34.217821 23761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 382bc085a55b4a5cbe6457119a6ef7fa [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:34.216117 23759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 382bc085a55b4a5cbe6457119a6ef7fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "382bc085a55b4a5cbe6457119a6ef7fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "382bc085a55b4a5cbe6457119a6ef7fa" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 44163 } } }
I20260807 18:14:34.219264 23759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 382bc085a55b4a5cbe6457119a6ef7fa [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:34.223551 23773 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:14:34.230446 23773 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:14:34.244032 23773 catalog_manager.cc:1347] Generated new cluster ID: 1e6e90a547fa41208a1f2db23ba37f6b
I20260807 18:14:34.244189 23773 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:14:34.260962 23773 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:14:34.262650 23773 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:14:34.278563 23773 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 382bc085a55b4a5cbe6457119a6ef7fa: Generated new TSK 0
I20260807 18:14:34.279605 23773 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.110.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
I20260807 18:14:34.364390 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.1:0
--local_ip_for_outbound_sockets=127.17.110.1
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:44163
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:41733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:14:34.753612 23788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:34.754010 23788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:34.754192 23788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:34.761931 23788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:34.762148 23788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:34.762336 23788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.1
I20260807 18:14:34.773061 23788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:41733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:44163
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.1
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:34.774590 23788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:34.776108 23788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:34.791664 23798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:34.791659 23793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:34.791685 23795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:34.792451 23788 server_base.cc:1034] running on GCE node
I20260807 18:14:34.793200 23788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:34.794296 23788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:34.795552 23788 hybrid_clock.cc:648] HybridClock initialized: now 1786126474795509 us; error 49 us; skew 500 ppm
I20260807 18:14:34.795924 23788 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:34 dist-test-slave-39l8 krb5kdc[23640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126474, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:34.799468 23788 init.cc:377] Logged in from keytab as kudu/127.17.110.1@KRBTEST.COM (short username kudu)
I20260807 18:14:34.800865 23788 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:34.801019 23788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:34.801365 23788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:34.804651 23788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-0/data/instance:
uuid: "805662f8bd66449298f170ca4fd8b029"
format_stamp: "Formatted at 2026-08-07 18:14:34 on dist-test-slave-39l8"
server_key: "0f3607caa33b22f951d7673aa62acf6b"
server_key_iv: "f10aee145f842e25d8d3b9a0e8666b35"
server_key_version: "encryptionkey@0"
I20260807 18:14:34.805725 23788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance:
uuid: "805662f8bd66449298f170ca4fd8b029"
format_stamp: "Formatted at 2026-08-07 18:14:34 on dist-test-slave-39l8"
server_key: "0f3607caa33b22f951d7673aa62acf6b"
server_key_iv: "f10aee145f842e25d8d3b9a0e8666b35"
server_key_version: "encryptionkey@0"
I20260807 18:14:34.811956 23788 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 18:14:34.816357 23808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:34.817934 23788 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:14:34.818177 23788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-0/wal
uuid: "805662f8bd66449298f170ca4fd8b029"
format_stamp: "Formatted at 2026-08-07 18:14:34 on dist-test-slave-39l8"
server_key: "0f3607caa33b22f951d7673aa62acf6b"
server_key_iv: "f10aee145f842e25d8d3b9a0e8666b35"
server_key_version: "encryptionkey@0"
I20260807 18:14:34.818447 23788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:34.853713 23788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:34.859462 23788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:34.859818 23788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:34.861004 23788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:14:34.862520 23788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:14:34.862661 23788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:34.862838 23788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:14:34.862936 23788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:34.899986 23788 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.1:34355
I20260807 18:14:34.900049 23953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.1:34355 every 8 connection(s)
I20260807 18:14:34.900965 23788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
I20260807 18:14:34.908213 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 23788
I20260807 18:14:34.908458 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance
I20260807 18:14:34.909150 17848 external_mini_cluster.cc:1428] Setting key 251c2de0891108d37bfd4d108c00e541
Aug 07 18:14:34 dist-test-slave-39l8 krb5kdc[23640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126474, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:34.923934 23956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:34.903905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:52569 (local address 127.17.110.62:44163)
0807 18:14:34.904479 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:34.904484 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:34.905328 (+ 844us) server_negotiation.cc:407] Connection header received
0807 18:14:34.907528 (+ 2200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:34.907542 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:34.907636 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:34.907764 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:34.909631 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:34.910525 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:34.912288 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:34.912703 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:34.912847 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:34.916905 (+ 4058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:34.916911 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:34.916915 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:34.916954 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:34.919900 (+ 2946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:34.921034 (+ 1134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:34.921039 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:34.921042 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:34.921196 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:34.921799 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:34.921803 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:34.921806 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:34.922095 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:34.922219 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:34.923372 (+ 1153us) server_negotiation.cc:299] Negotiation successful
0807 18:14:34.923647 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":355,"threads_started":1}
I20260807 18:14:34.925757 23958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:34.904364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:44163 (local address 127.17.110.1:52569)
0807 18:14:34.905155 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:34.905195 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:34.907228 (+ 2033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:34.908049 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:34.908057 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:34.908736 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:34.909416 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:34.909437 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:34.910769 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:34.910774 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:34.912109 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:34.912123 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:34.912966 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:34.912974 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:34.913240 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:34.914082 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:34.914108 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:34.916614 (+ 2506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:34.920110 (+ 3496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:34.920121 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:34.920151 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:34.920847 (+ 696us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:34.921328 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:34.921336 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:34.921343 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:34.921638 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:34.922269 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:34.922281 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:34.922890 (+ 609us) client_negotiation.cc:769] Sending connection context
0807 18:14:34.923248 (+ 358us) client_negotiation.cc:240] Negotiation successful
0807 18:14:34.923947 (+ 699us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":297,"threads_started":1}
I20260807 18:14:34.927709 23954 heartbeater.cc:347] Connected to a master server at 127.17.110.62:44163
I20260807 18:14:34.928074 23954 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:34.928802 23954 heartbeater.cc:511] Master 127.17.110.62:44163 requested a full tablet report, sending...
I20260807 18:14:34.931303 23699 ts_manager.cc:194] Registered new tserver with Master: 805662f8bd66449298f170ca4fd8b029 (127.17.110.1:34355)
I20260807 18:14:34.932878 23699 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.1@KRBTEST.COM'} at 127.17.110.1:52569
WARNING: no policy specified for kudu/127.17.110.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
I20260807 18:14:34.966866 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.2:0
--local_ip_for_outbound_sockets=127.17.110.2
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:44163
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:41733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:14:35.325662 23961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:35.326038 23961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:35.326225 23961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:35.333871 23961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:35.334064 23961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:35.334257 23961 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.2
I20260807 18:14:35.344565 23961 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:41733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:44163
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.2
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:35.346014 23961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:35.347442 23961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:35.361868 23973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:35.361899 23970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:35.361872 23969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:35.363204 23961 server_base.cc:1034] running on GCE node
I20260807 18:14:35.363942 23961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:35.365180 23961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:35.366468 23961 hybrid_clock.cc:648] HybridClock initialized: now 1786126475366380 us; error 83 us; skew 500 ppm
I20260807 18:14:35.367002 23961 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:35 dist-test-slave-39l8 krb5kdc[23640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126475, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:35.371345 23961 init.cc:377] Logged in from keytab as kudu/127.17.110.2@KRBTEST.COM (short username kudu)
I20260807 18:14:35.372989 23961 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:35.373180 23961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:35.373584 23961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:35.377918 23961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-1/data/instance:
uuid: "4989ef67f500461aa0736cd2d072a3a5"
format_stamp: "Formatted at 2026-08-07 18:14:35 on dist-test-slave-39l8"
server_key: "2da56dfcaa7f52b3059dfe31d86922af"
server_key_iv: "8e8300309afdfb2c2770ccac4b8f917e"
server_key_version: "encryptionkey@0"
I20260807 18:14:35.379426 23961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance:
uuid: "4989ef67f500461aa0736cd2d072a3a5"
format_stamp: "Formatted at 2026-08-07 18:14:35 on dist-test-slave-39l8"
server_key: "2da56dfcaa7f52b3059dfe31d86922af"
server_key_iv: "8e8300309afdfb2c2770ccac4b8f917e"
server_key_version: "encryptionkey@0"
I20260807 18:14:35.387334 23961 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260807 18:14:35.392181 23984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:35.395141 23961 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:14:35.395421 23961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-1/wal
uuid: "4989ef67f500461aa0736cd2d072a3a5"
format_stamp: "Formatted at 2026-08-07 18:14:35 on dist-test-slave-39l8"
server_key: "2da56dfcaa7f52b3059dfe31d86922af"
server_key_iv: "8e8300309afdfb2c2770ccac4b8f917e"
server_key_version: "encryptionkey@0"
I20260807 18:14:35.395705 23961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:35.415910 23961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:35.422013 23961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:35.422514 23961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:35.423815 23961 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:14:35.425999 23961 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:14:35.426234 23961 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:35.426391 23961 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:14:35.426514 23961 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:35.466851 23961 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.2:35587
I20260807 18:14:35.466946 24137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.2:35587 every 8 connection(s)
I20260807 18:14:35.467813 23961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
I20260807 18:14:35.473459 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 23961
I20260807 18:14:35.473701 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance
I20260807 18:14:35.474346 17848 external_mini_cluster.cc:1428] Setting key 078f47d6805578992fb7d41bf2430885
Aug 07 18:14:35 dist-test-slave-39l8 krb5kdc[23640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126475, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:35.492623 24141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:35.470496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:55665 (local address 127.17.110.62:44163)
0807 18:14:35.471027 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:35.471034 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:35.472051 (+ 1017us) server_negotiation.cc:407] Connection header received
0807 18:14:35.474048 (+ 1997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:35.474055 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:35.474155 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:35.474305 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:35.476323 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:35.477120 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:35.479960 (+ 2840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:35.480494 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:35.480633 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:35.485431 (+ 4798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:35.485441 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:35.485450 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:35.485507 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:35.488530 (+ 3023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:35.489756 (+ 1226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:35.489762 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:35.489764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:35.489909 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:35.490457 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:35.490461 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:35.490463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:35.490863 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:35.490991 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:35.492111 (+ 1120us) server_negotiation.cc:299] Negotiation successful
0807 18:14:35.492314 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":265,"threads_started":1}
I20260807 18:14:35.494264 24142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:35.471075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:44163 (local address 127.17.110.2:55665)
0807 18:14:35.471845 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:35.471894 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:35.473681 (+ 1787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:35.474583 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:35.474597 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:35.475297 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:35.475979 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:35.475999 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:35.477300 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:35.477307 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:35.479751 (+ 2444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:35.479772 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:35.480774 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:35.480785 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:35.481139 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:35.481997 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:35.482058 (+ 61us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:35.485098 (+ 3040us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:35.488727 (+ 3629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:35.488742 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:35.488765 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:35.489560 (+ 795us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:35.490039 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:35.490047 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:35.490051 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:35.490316 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:35.491004 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:35.491016 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:35.491594 (+ 578us) client_negotiation.cc:769] Sending connection context
0807 18:14:35.491965 (+ 371us) client_negotiation.cc:240] Negotiation successful
0807 18:14:35.492647 (+ 682us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":302,"threads_started":1}
I20260807 18:14:35.496102 24138 heartbeater.cc:347] Connected to a master server at 127.17.110.62:44163
I20260807 18:14:35.496469 24138 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:35.497368 24138 heartbeater.cc:511] Master 127.17.110.62:44163 requested a full tablet report, sending...
I20260807 18:14:35.499182 23699 ts_manager.cc:194] Registered new tserver with Master: 4989ef67f500461aa0736cd2d072a3a5 (127.17.110.2:35587)
I20260807 18:14:35.500564 23699 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.2@KRBTEST.COM'} at 127.17.110.2:55665
WARNING: no policy specified for kudu/127.17.110.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
I20260807 18:14:35.534930 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.3:0
--local_ip_for_outbound_sockets=127.17.110.3
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:44163
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:41733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:14:35.895205 24146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:35.895601 24146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:35.895802 24146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:35.904551 24146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:35.904733 24146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:35.904915 24146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.3
I20260807 18:14:35.916405 24146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:41733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:44163
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.3
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:35.917935 24146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:35.919516 24146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:35.933727 24153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:35.933758 24156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:35.933730 24154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:35.934913 24146 server_base.cc:1034] running on GCE node
I20260807 18:14:35.935614 24146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:35.936412 23954 heartbeater.cc:503] Master 127.17.110.62:44163 was elected leader, sending a full tablet report...
I20260807 18:14:35.936789 24146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:35.938028 24146 hybrid_clock.cc:648] HybridClock initialized: now 1786126475937980 us; error 57 us; skew 500 ppm
I20260807 18:14:35.938454 24146 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:35 dist-test-slave-39l8 krb5kdc[23640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126475, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:35.942167 24146 init.cc:377] Logged in from keytab as kudu/127.17.110.3@KRBTEST.COM (short username kudu)
I20260807 18:14:35.943595 24146 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:35.943702 24146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:35.944121 24146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:35.947544 24146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-2/data/instance:
uuid: "a98410440db74f4fb464af07dbb2b59a"
format_stamp: "Formatted at 2026-08-07 18:14:35 on dist-test-slave-39l8"
server_key: "df6357ef5179662cf10d8f0a511112dc"
server_key_iv: "c7e1c12198f98e8e403dc94994508353"
server_key_version: "encryptionkey@0"
I20260807 18:14:35.948483 24146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance:
uuid: "a98410440db74f4fb464af07dbb2b59a"
format_stamp: "Formatted at 2026-08-07 18:14:35 on dist-test-slave-39l8"
server_key: "df6357ef5179662cf10d8f0a511112dc"
server_key_iv: "c7e1c12198f98e8e403dc94994508353"
server_key_version: "encryptionkey@0"
I20260807 18:14:35.954854 24146 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260807 18:14:35.959563 24163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:35.961404 24146 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:14:35.961649 24146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-2/wal
uuid: "a98410440db74f4fb464af07dbb2b59a"
format_stamp: "Formatted at 2026-08-07 18:14:35 on dist-test-slave-39l8"
server_key: "df6357ef5179662cf10d8f0a511112dc"
server_key_iv: "c7e1c12198f98e8e403dc94994508353"
server_key_version: "encryptionkey@0"
I20260807 18:14:35.961975 24146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:35.988195 24146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:35.995223 24146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:35.995815 24146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:35.997426 24146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:14:35.999471 24146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:14:35.999599 24146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:35.999687 24146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:14:35.999748 24146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:36.040462 24146 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.3:35543
I20260807 18:14:36.040549 24315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.3:35543 every 8 connection(s)
I20260807 18:14:36.041447 24146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
I20260807 18:14:36.041903 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 24146
I20260807 18:14:36.042146 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance
I20260807 18:14:36.042793 17848 external_mini_cluster.cc:1428] Setting key f5497dc57b534c06db27a5207b3b38f6
Aug 07 18:14:36 dist-test-slave-39l8 krb5kdc[23640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126475, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:36.064425 24318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:36.044493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:54101 (local address 127.17.110.62:44163)
0807 18:14:36.045051 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:36.045058 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:36.046267 (+ 1209us) server_negotiation.cc:407] Connection header received
0807 18:14:36.048320 (+ 2053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:36.048324 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:36.048412 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:36.048510 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:36.050335 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.051231 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.053019 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.053444 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.053556 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:36.057796 (+ 4240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:36.057806 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:36.057815 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:36.057865 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:36.061149 (+ 3284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:36.062021 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:36.062026 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:36.062028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:36.062172 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:36.062639 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:36.062644 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:36.062646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:36.063046 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:36.063175 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:36.063933 (+ 758us) server_negotiation.cc:299] Negotiation successful
0807 18:14:36.064119 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":288,"threads_started":1}
I20260807 18:14:36.065522 24319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:36.045193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:44163 (local address 127.17.110.3:54101)
0807 18:14:36.046073 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:36.046126 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:36.048004 (+ 1878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:36.048840 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:36.048850 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:36.049460 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:36.050129 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:36.050150 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.051433 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.051437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:36.052841 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:36.052857 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.053679 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.053685 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:36.053941 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:36.054786 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:36.054817 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:36.057462 (+ 2645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:36.061329 (+ 3867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:36.061339 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:36.061356 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:36.061895 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:36.062285 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:36.062288 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:36.062291 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:36.062498 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:36.063177 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:36.063185 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:36.063559 (+ 374us) client_negotiation.cc:769] Sending connection context
0807 18:14:36.063824 (+ 265us) client_negotiation.cc:240] Negotiation successful
0807 18:14:36.064270 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":328,"threads_started":1}
I20260807 18:14:36.067349 24316 heartbeater.cc:347] Connected to a master server at 127.17.110.62:44163
I20260807 18:14:36.067667 24316 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:36.068344 24316 heartbeater.cc:511] Master 127.17.110.62:44163 requested a full tablet report, sending...
I20260807 18:14:36.070030 23699 ts_manager.cc:194] Registered new tserver with Master: a98410440db74f4fb464af07dbb2b59a (127.17.110.3:35543)
I20260807 18:14:36.071255 23699 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.3@KRBTEST.COM'} at 127.17.110.3:54101
I20260807 18:14:36.072453 17848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:14:36.092660 24318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:36.078445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60586 (local address 127.17.110.62:44163)
0807 18:14:36.078650 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:36.078655 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:36.079187 (+ 532us) server_negotiation.cc:407] Connection header received
0807 18:14:36.079355 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:36.079359 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:36.079453 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:36.079557 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:36.081284 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.082031 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.083482 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.083912 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.084049 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:36.086824 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:36.086830 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:36.086834 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:36.086879 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:36.089825 (+ 2946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:36.090675 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:36.090679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:36.090681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:36.090863 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:36.091300 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:36.091303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:36.091305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:36.091644 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:36.091847 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:36.092187 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 18:14:36.092412 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:14:36.097342 23699 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60586:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:14:36.104684 23699 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:14:36.158022 24337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:36.120585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46844 (local address 127.17.110.1:34355)
0807 18:14:36.144229 (+ 23644us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:36.144245 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:36.147818 (+ 3573us) server_negotiation.cc:407] Connection header received
0807 18:14:36.147946 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:36.147954 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:36.148239 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:36.148450 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:14:36.149776 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.151469 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.154983 (+ 3514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.156545 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.156733 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:36.157024 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:36.157373 (+ 349us) server_negotiation.cc:299] Negotiation successful
0807 18:14:36.157583 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23326,"thread_start_us":23677,"threads_started":1}
I20260807 18:14:36.158233 24330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:36.119478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:34355 (local address 127.0.0.1:46844)
0807 18:14:36.147402 (+ 27924us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:36.147429 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:36.147594 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:36.148740 (+ 1146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:36.148766 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:36.148802 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:14:36.149510 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:36.149527 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.151706 (+ 2179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.151716 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:36.154357 (+ 2641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:36.154374 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.156787 (+ 2413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.156795 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:36.157072 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:36.157134 (+ 62us) client_negotiation.cc:769] Sending connection context
0807 18:14:36.157328 (+ 194us) client_negotiation.cc:240] Negotiation successful
0807 18:14:36.157582 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27768,"mutex_wait_us":231,"thread_start_us":28229,"threads_started":1}
I20260807 18:14:36.158867 24332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:36.119609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37754 (local address 127.17.110.3:35543)
0807 18:14:36.147676 (+ 28067us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:36.147690 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:36.147746 (+ 56us) server_negotiation.cc:407] Connection header received
0807 18:14:36.147873 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:36.147881 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:36.148200 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:36.148440 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:14:36.149754 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.151596 (+ 1842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.155103 (+ 3507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.157067 (+ 1964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.157206 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:36.157464 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:36.158227 (+ 763us) server_negotiation.cc:299] Negotiation successful
0807 18:14:36.158436 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27802,"thread_start_us":27716,"threads_started":1}
I20260807 18:14:36.159169 24335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:36.119189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:35543 (local address 127.0.0.1:37754)
0807 18:14:36.145431 (+ 26242us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:36.145530 (+ 99us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:36.145930 (+ 400us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:36.148612 (+ 2682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:36.148624 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:36.148664 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:14:36.149392 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:36.149429 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.151793 (+ 2364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.151800 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:36.154875 (+ 3075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:36.154892 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.157629 (+ 2737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.157635 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:36.157813 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:36.157846 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 18:14:36.157944 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 18:14:36.158085 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25812,"mutex_wait_us":156,"thread_start_us":27855,"threads_started":1}
I20260807 18:14:36.160989 24338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:36.121179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47458 (local address 127.17.110.2:35587)
0807 18:14:36.146977 (+ 25798us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:36.146992 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:36.148267 (+ 1275us) server_negotiation.cc:407] Connection header received
0807 18:14:36.148395 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:36.148404 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:36.148631 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:36.148837 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:14:36.149893 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.151596 (+ 1703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.155103 (+ 3507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.156665 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.156831 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:36.157033 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:36.160377 (+ 3344us) server_negotiation.cc:299] Negotiation successful
0807 18:14:36.160599 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25553,"thread_start_us":25315,"threads_started":1}
I20260807 18:14:36.161950 24331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:36.120329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:35587 (local address 127.0.0.1:47458)
0807 18:14:36.147987 (+ 27658us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:36.148006 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:36.148121 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:36.148894 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:36.148899 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:36.148927 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:14:36.149677 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:36.149693 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.151795 (+ 2102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.151800 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:36.154876 (+ 3076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:36.154892 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.157253 (+ 2361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.157260 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:36.157456 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:36.157496 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 18:14:36.157600 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 18:14:36.157764 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27522,"mutex_wait_us":101,"thread_start_us":24493,"threads_started":1}
I20260807 18:14:36.163837 23875 tablet_service.cc:1467] Processing CreateTablet for tablet 9f1c9890ceee42b985d560586e472fa7 (DEFAULT_TABLE table=test-table [id=08fa76a961ee479b8fac3b4b53e089ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:14:36.165652 23875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f1c9890ceee42b985d560586e472fa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:36.164978 24225 tablet_service.cc:1467] Processing CreateTablet for tablet 9f1c9890ceee42b985d560586e472fa7 (DEFAULT_TABLE table=test-table [id=08fa76a961ee479b8fac3b4b53e089ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:14:36.165843 24052 tablet_service.cc:1467] Processing CreateTablet for tablet 9f1c9890ceee42b985d560586e472fa7 (DEFAULT_TABLE table=test-table [id=08fa76a961ee479b8fac3b4b53e089ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:14:36.166646 24225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f1c9890ceee42b985d560586e472fa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:36.167274 24052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f1c9890ceee42b985d560586e472fa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:36.188506 24339 tablet_bootstrap.cc:492] T 9f1c9890ceee42b985d560586e472fa7 P 805662f8bd66449298f170ca4fd8b029: Bootstrap starting.
I20260807 18:14:36.190045 24341 tablet_bootstrap.cc:492] T 9f1c9890ceee42b985d560586e472fa7 P 4989ef67f500461aa0736cd2d072a3a5: Bootstrap starting.
I20260807 18:14:36.190315 24340 tablet_bootstrap.cc:492] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a: Bootstrap starting.
I20260807 18:14:36.192773 24339 tablet_bootstrap.cc:654] T 9f1c9890ceee42b985d560586e472fa7 P 805662f8bd66449298f170ca4fd8b029: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:36.193447 24340 tablet_bootstrap.cc:654] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:36.193804 24341 tablet_bootstrap.cc:654] T 9f1c9890ceee42b985d560586e472fa7 P 4989ef67f500461aa0736cd2d072a3a5: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:36.194885 24340 log.cc:824] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:14:36.195015 24339 log.cc:824] T 9f1c9890ceee42b985d560586e472fa7 P 805662f8bd66449298f170ca4fd8b029: Log is configured to *not* fsync() on all Append() calls
I20260807 18:14:36.195145 24341 log.cc:824] T 9f1c9890ceee42b985d560586e472fa7 P 4989ef67f500461aa0736cd2d072a3a5: Log is configured to *not* fsync() on all Append() calls
I20260807 18:14:36.198494 24341 tablet_bootstrap.cc:492] T 9f1c9890ceee42b985d560586e472fa7 P 4989ef67f500461aa0736cd2d072a3a5: No bootstrap required, opened a new log
I20260807 18:14:36.198647 24339 tablet_bootstrap.cc:492] T 9f1c9890ceee42b985d560586e472fa7 P 805662f8bd66449298f170ca4fd8b029: No bootstrap required, opened a new log
I20260807 18:14:36.198647 24340 tablet_bootstrap.cc:492] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a: No bootstrap required, opened a new log
I20260807 18:14:36.198874 24341 ts_tablet_manager.cc:1397] T 9f1c9890ceee42b985d560586e472fa7 P 4989ef67f500461aa0736cd2d072a3a5: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260807 18:14:36.198971 24339 ts_tablet_manager.cc:1397] T 9f1c9890ceee42b985d560586e472fa7 P 805662f8bd66449298f170ca4fd8b029: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260807 18:14:36.198971 24340 ts_tablet_manager.cc:1397] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260807 18:14:36.207629 24341 raft_consensus.cc:348] T 9f1c9890ceee42b985d560586e472fa7 P 4989ef67f500461aa0736cd2d072a3a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a98410440db74f4fb464af07dbb2b59a" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 35543 } } peers { permanent_uuid: "805662f8bd66449298f170ca4fd8b029" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 34355 } } peers { permanent_uuid: "4989ef67f500461aa0736cd2d072a3a5" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35587 } }
I20260807 18:14:36.208122 24341 raft_consensus.cc:374] T 9f1c9890ceee42b985d560586e472fa7 P 4989ef67f500461aa0736cd2d072a3a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:36.208223 24341 raft_consensus.cc:729] T 9f1c9890ceee42b985d560586e472fa7 P 4989ef67f500461aa0736cd2d072a3a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4989ef67f500461aa0736cd2d072a3a5, State: Initialized, Role: FOLLOWER
I20260807 18:14:36.208930 24341 consensus_queue.cc:260] T 9f1c9890ceee42b985d560586e472fa7 P 4989ef67f500461aa0736cd2d072a3a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a98410440db74f4fb464af07dbb2b59a" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 35543 } } peers { permanent_uuid: "805662f8bd66449298f170ca4fd8b029" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 34355 } } peers { permanent_uuid: "4989ef67f500461aa0736cd2d072a3a5" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35587 } }
I20260807 18:14:36.210783 24340 raft_consensus.cc:348] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a98410440db74f4fb464af07dbb2b59a" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 35543 } } peers { permanent_uuid: "805662f8bd66449298f170ca4fd8b029" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 34355 } } peers { permanent_uuid: "4989ef67f500461aa0736cd2d072a3a5" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35587 } }
I20260807 18:14:36.210793 24339 raft_consensus.cc:348] T 9f1c9890ceee42b985d560586e472fa7 P 805662f8bd66449298f170ca4fd8b029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a98410440db74f4fb464af07dbb2b59a" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 35543 } } peers { permanent_uuid: "805662f8bd66449298f170ca4fd8b029" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 34355 } } peers { permanent_uuid: "4989ef67f500461aa0736cd2d072a3a5" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35587 } }
I20260807 18:14:36.211289 24339 raft_consensus.cc:374] T 9f1c9890ceee42b985d560586e472fa7 P 805662f8bd66449298f170ca4fd8b029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:36.211290 24340 raft_consensus.cc:374] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:36.211400 24339 raft_consensus.cc:729] T 9f1c9890ceee42b985d560586e472fa7 P 805662f8bd66449298f170ca4fd8b029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 805662f8bd66449298f170ca4fd8b029, State: Initialized, Role: FOLLOWER
I20260807 18:14:36.211395 24340 raft_consensus.cc:729] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a98410440db74f4fb464af07dbb2b59a, State: Initialized, Role: FOLLOWER
I20260807 18:14:36.212239 24339 consensus_queue.cc:260] T 9f1c9890ceee42b985d560586e472fa7 P 805662f8bd66449298f170ca4fd8b029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a98410440db74f4fb464af07dbb2b59a" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 35543 } } peers { permanent_uuid: "805662f8bd66449298f170ca4fd8b029" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 34355 } } peers { permanent_uuid: "4989ef67f500461aa0736cd2d072a3a5" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35587 } }
I20260807 18:14:36.212261 24340 consensus_queue.cc:260] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a98410440db74f4fb464af07dbb2b59a" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 35543 } } peers { permanent_uuid: "805662f8bd66449298f170ca4fd8b029" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 34355 } } peers { permanent_uuid: "4989ef67f500461aa0736cd2d072a3a5" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35587 } }
I20260807 18:14:36.212620 24138 heartbeater.cc:503] Master 127.17.110.62:44163 was elected leader, sending a full tablet report...
I20260807 18:14:36.213388 24341 ts_tablet_manager.cc:1428] T 9f1c9890ceee42b985d560586e472fa7 P 4989ef67f500461aa0736cd2d072a3a5: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260807 18:14:36.214637 24316 heartbeater.cc:503] Master 127.17.110.62:44163 was elected leader, sending a full tablet report...
I20260807 18:14:36.215992 24339 ts_tablet_manager.cc:1428] T 9f1c9890ceee42b985d560586e472fa7 P 805662f8bd66449298f170ca4fd8b029: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
I20260807 18:14:36.217654 24340 ts_tablet_manager.cc:1428] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a: Time spent starting tablet: real 0.019s user 0.011s sys 0.004s
W20260807 18:14:36.219871 24140 tablet.cc:2369] T 9f1c9890ceee42b985d560586e472fa7 P 4989ef67f500461aa0736cd2d072a3a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:14:36.293409 24317 tablet.cc:2369] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:14:36.403755 23955 tablet.cc:2369] T 9f1c9890ceee42b985d560586e472fa7 P 805662f8bd66449298f170ca4fd8b029: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:14:36.476110 24349 raft_consensus.cc:482] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:14:36.476461 24349 raft_consensus.cc:504] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a98410440db74f4fb464af07dbb2b59a" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 35543 } } peers { permanent_uuid: "805662f8bd66449298f170ca4fd8b029" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 34355 } } peers { permanent_uuid: "4989ef67f500461aa0736cd2d072a3a5" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35587 } }
I20260807 18:14:36.478274 24349 leader_election.cc:288] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 805662f8bd66449298f170ca4fd8b029 (127.17.110.1:34355), 4989ef67f500461aa0736cd2d072a3a5 (127.17.110.2:35587)
I20260807 18:14:36.484017 24319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:36.478491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:34355 (local address 127.17.110.3:43323)
0807 18:14:36.478741 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:36.478765 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:36.478911 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:36.479438 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:36.479442 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:36.479476 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:14:36.479638 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:36.479646 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.480687 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.480693 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:36.482026 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:36.482039 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.483249 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.483255 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:36.483454 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:36.483479 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 18:14:36.483594 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 18:14:36.483729 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 18:14:36.484020 24337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:36.478663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:43323 (local address 127.17.110.1:34355)
0807 18:14:36.478957 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:36.478965 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:36.478989 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:14:36.479068 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:36.479073 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:36.479141 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:36.479254 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:14:36.479803 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.480537 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.482212 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.483078 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.483255 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:36.483343 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:36.483630 (+ 287us) server_negotiation.cc:299] Negotiation successful
0807 18:14:36.483769 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146}
I20260807 18:14:36.485047 24338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:36.479114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:40837 (local address 127.17.110.2:35587)
0807 18:14:36.479341 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:36.479347 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:36.479798 (+ 451us) server_negotiation.cc:407] Connection header received
0807 18:14:36.480038 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:36.480042 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:36.480107 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:36.480212 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:14:36.480780 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.481510 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.483232 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.483996 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.484241 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:36.484330 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:36.484761 (+ 431us) server_negotiation.cc:299] Negotiation successful
0807 18:14:36.484864 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 18:14:36.485205 24351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:36.478854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:35587 (local address 127.17.110.3:40837)
0807 18:14:36.479676 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:36.479693 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:36.479888 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:36.480232 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:36.480236 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:36.480262 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:14:36.480617 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:36.480626 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.481688 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.481692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:36.483064 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:36.483077 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.484371 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.484378 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:36.484543 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:36.484583 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 18:14:36.484706 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 18:14:36.484853 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"thread_start_us":473,"threads_started":1}
I20260807 18:14:36.485288 23900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f1c9890ceee42b985d560586e472fa7" candidate_uuid: "a98410440db74f4fb464af07dbb2b59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "805662f8bd66449298f170ca4fd8b029" is_pre_election: true
I20260807 18:14:36.485934 23900 raft_consensus.cc:2454] T 9f1c9890ceee42b985d560586e472fa7 P 805662f8bd66449298f170ca4fd8b029 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a98410440db74f4fb464af07dbb2b59a in term 0.
I20260807 18:14:36.486366 24076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f1c9890ceee42b985d560586e472fa7" candidate_uuid: "a98410440db74f4fb464af07dbb2b59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4989ef67f500461aa0736cd2d072a3a5" is_pre_election: true
I20260807 18:14:36.486696 24165 leader_election.cc:302] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 805662f8bd66449298f170ca4fd8b029, a98410440db74f4fb464af07dbb2b59a; no voters:
I20260807 18:14:36.487030 24076 raft_consensus.cc:2454] T 9f1c9890ceee42b985d560586e472fa7 P 4989ef67f500461aa0736cd2d072a3a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a98410440db74f4fb464af07dbb2b59a in term 0.
I20260807 18:14:36.487176 24349 raft_consensus.cc:2781] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:14:36.487361 24349 raft_consensus.cc:482] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:14:36.487478 24349 raft_consensus.cc:3037] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:36.489899 24349 raft_consensus.cc:504] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a98410440db74f4fb464af07dbb2b59a" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 35543 } } peers { permanent_uuid: "805662f8bd66449298f170ca4fd8b029" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 34355 } } peers { permanent_uuid: "4989ef67f500461aa0736cd2d072a3a5" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35587 } }
I20260807 18:14:36.490593 24349 leader_election.cc:288] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a [CANDIDATE]: Term 1 election: Requested vote from peers 805662f8bd66449298f170ca4fd8b029 (127.17.110.1:34355), 4989ef67f500461aa0736cd2d072a3a5 (127.17.110.2:35587)
I20260807 18:14:36.491189 24076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f1c9890ceee42b985d560586e472fa7" candidate_uuid: "a98410440db74f4fb464af07dbb2b59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4989ef67f500461aa0736cd2d072a3a5"
I20260807 18:14:36.491201 23900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f1c9890ceee42b985d560586e472fa7" candidate_uuid: "a98410440db74f4fb464af07dbb2b59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "805662f8bd66449298f170ca4fd8b029"
I20260807 18:14:36.491430 24076 raft_consensus.cc:3037] T 9f1c9890ceee42b985d560586e472fa7 P 4989ef67f500461aa0736cd2d072a3a5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:36.491457 23900 raft_consensus.cc:3037] T 9f1c9890ceee42b985d560586e472fa7 P 805662f8bd66449298f170ca4fd8b029 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:36.493780 23900 raft_consensus.cc:2454] T 9f1c9890ceee42b985d560586e472fa7 P 805662f8bd66449298f170ca4fd8b029 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a98410440db74f4fb464af07dbb2b59a in term 1.
I20260807 18:14:36.493793 24076 raft_consensus.cc:2454] T 9f1c9890ceee42b985d560586e472fa7 P 4989ef67f500461aa0736cd2d072a3a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a98410440db74f4fb464af07dbb2b59a in term 1.
I20260807 18:14:36.494354 24165 leader_election.cc:302] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 805662f8bd66449298f170ca4fd8b029, a98410440db74f4fb464af07dbb2b59a; no voters:
I20260807 18:14:36.494693 24349 raft_consensus.cc:2781] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:14:36.495234 24349 raft_consensus.cc:686] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a [term 1 LEADER]: Becoming Leader. State: Replica: a98410440db74f4fb464af07dbb2b59a, State: Running, Role: LEADER
I20260807 18:14:36.495765 24349 consensus_queue.cc:237] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a98410440db74f4fb464af07dbb2b59a" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 35543 } } peers { permanent_uuid: "805662f8bd66449298f170ca4fd8b029" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 34355 } } peers { permanent_uuid: "4989ef67f500461aa0736cd2d072a3a5" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35587 } }
I20260807 18:14:36.501127 23699 catalog_manager.cc:5495] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a reported cstate change: term changed from 0 to 1, leader changed from <none> to a98410440db74f4fb464af07dbb2b59a (127.17.110.3). New cstate: current_term: 1 leader_uuid: "a98410440db74f4fb464af07dbb2b59a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a98410440db74f4fb464af07dbb2b59a" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 35543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "805662f8bd66449298f170ca4fd8b029" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 34355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4989ef67f500461aa0736cd2d072a3a5" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35587 } health_report { overall_health: UNKNOWN } } }
I20260807 18:14:36.543727 24332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:36.537984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37766 (local address 127.17.110.3:35543)
0807 18:14:36.538219 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:36.538225 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:36.538248 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:14:36.538427 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:36.538432 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:36.538531 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:36.538702 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:14:36.539165 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.539903 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.541284 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.541711 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.541866 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:36.542410 (+ 544us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:14:36.543060 (+ 650us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:14:36.543232 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:36.543342 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 18:14:36.543435 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:14:36.553068 23900 raft_consensus.cc:1261] T 9f1c9890ceee42b985d560586e472fa7 P 805662f8bd66449298f170ca4fd8b029 [term 1 FOLLOWER]: Refusing update from remote peer a98410440db74f4fb464af07dbb2b59a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:14:36.553073 24076 raft_consensus.cc:1261] T 9f1c9890ceee42b985d560586e472fa7 P 4989ef67f500461aa0736cd2d072a3a5 [term 1 FOLLOWER]: Refusing update from remote peer a98410440db74f4fb464af07dbb2b59a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:14:36.554306 24353 consensus_queue.cc:1035] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4989ef67f500461aa0736cd2d072a3a5" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 35587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:14:36.554783 24349 consensus_queue.cc:1035] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "805662f8bd66449298f170ca4fd8b029" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 34355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:14:36.569947 24358 mvcc.cc:205] Tried to move back new op lower bound from 7315974047948967936 to 7315974047730786304. Current Snapshot: MvccSnapshot[applied={T|T < 7315974047948967936}]
I20260807 18:14:36.584959 23699 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:14:36.585323 23699 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:14:36.590689 23699 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 382bc085a55b4a5cbe6457119a6ef7fa: Sending DeleteTablet for 3 replicas of tablet 9f1c9890ceee42b985d560586e472fa7
I20260807 18:14:36.592276 23875 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f1c9890ceee42b985d560586e472fa7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:14:36 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:46844
I20260807 18:14:36.592263 24225 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f1c9890ceee42b985d560586e472fa7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:14:36 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:37754
I20260807 18:14:36.592345 24052 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f1c9890ceee42b985d560586e472fa7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:14:36 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:47458
I20260807 18:14:36.596077 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 23788
I20260807 18:14:36.605084 24377 tablet_replica.cc:330] T 9f1c9890ceee42b985d560586e472fa7 P 4989ef67f500461aa0736cd2d072a3a5: stopping tablet replica
I20260807 18:14:36.605464 24375 tablet_replica.cc:330] T 9f1c9890ceee42b985d560586e472fa7 P 805662f8bd66449298f170ca4fd8b029: stopping tablet replica
I20260807 18:14:36.605650 24377 raft_consensus.cc:2229] T 9f1c9890ceee42b985d560586e472fa7 P 4989ef67f500461aa0736cd2d072a3a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:14:36.606397 24377 raft_consensus.cc:2258] T 9f1c9890ceee42b985d560586e472fa7 P 4989ef67f500461aa0736cd2d072a3a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:14:36.606664 24375 raft_consensus.cc:2229] T 9f1c9890ceee42b985d560586e472fa7 P 805662f8bd66449298f170ca4fd8b029 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:14:36.607308 24375 raft_consensus.cc:2258] T 9f1c9890ceee42b985d560586e472fa7 P 805662f8bd66449298f170ca4fd8b029 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:14:36.609061 24377 ts_tablet_manager.cc:1905] T 9f1c9890ceee42b985d560586e472fa7 P 4989ef67f500461aa0736cd2d072a3a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:14:36.611407 24375 ts_tablet_manager.cc:1905] T 9f1c9890ceee42b985d560586e472fa7 P 805662f8bd66449298f170ca4fd8b029: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:14:36.614292 24372 tablet_replica.cc:330] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a: stopping tablet replica
I20260807 18:14:36.614869 24372 raft_consensus.cc:2229] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:14:36.615726 24372 raft_consensus.cc:2258] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:14:36.617561 24377 ts_tablet_manager.cc:1918] T 9f1c9890ceee42b985d560586e472fa7 P 4989ef67f500461aa0736cd2d072a3a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:14:36.617786 24377 log.cc:1196] T 9f1c9890ceee42b985d560586e472fa7 P 4989ef67f500461aa0736cd2d072a3a5: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-1/wal/wals/9f1c9890ceee42b985d560586e472fa7
I20260807 18:14:36.618448 24377 ts_tablet_manager.cc:1939] T 9f1c9890ceee42b985d560586e472fa7 P 4989ef67f500461aa0736cd2d072a3a5: Deleting consensus metadata
I20260807 18:14:36.620682 23682 catalog_manager.cc:4841] TS 4989ef67f500461aa0736cd2d072a3a5 (127.17.110.2:35587): tablet 9f1c9890ceee42b985d560586e472fa7 (table test-table [id=08fa76a961ee479b8fac3b4b53e089ed]) successfully deleted
I20260807 18:14:36.622983 24375 ts_tablet_manager.cc:1918] T 9f1c9890ceee42b985d560586e472fa7 P 805662f8bd66449298f170ca4fd8b029: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:14:36.623266 24375 log.cc:1196] T 9f1c9890ceee42b985d560586e472fa7 P 805662f8bd66449298f170ca4fd8b029: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-0/wal/wals/9f1c9890ceee42b985d560586e472fa7
Aug 07 18:14:36 dist-test-slave-39l8 krb5kdc[23640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.1@KRBTEST.COM
I20260807 18:14:36.623459 24372 ts_tablet_manager.cc:1905] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:14:36.623871 24375 ts_tablet_manager.cc:1939] T 9f1c9890ceee42b985d560586e472fa7 P 805662f8bd66449298f170ca4fd8b029: Deleting consensus metadata
I20260807 18:14:36.626052 23678 catalog_manager.cc:4841] TS 805662f8bd66449298f170ca4fd8b029 (127.17.110.1:34355): tablet 9f1c9890ceee42b985d560586e472fa7 (table test-table [id=08fa76a961ee479b8fac3b4b53e089ed]) successfully deleted
I20260807 18:14:36.631913 24372 ts_tablet_manager.cc:1918] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:14:36.632146 24372 log.cc:1196] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126429568977-17848-0/minicluster-data/ts-2/wal/wals/9f1c9890ceee42b985d560586e472fa7
I20260807 18:14:36.632490 24337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:36.598392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46848 (local address 127.17.110.1:34355)
0807 18:14:36.598658 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:36.598668 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:36.611322 (+ 12654us) server_negotiation.cc:407] Connection header received
0807 18:14:36.611961 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:36.611969 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:36.612106 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:36.612202 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:36.614314 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.615405 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.617554 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.618280 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.618398 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:36.625216 (+ 6818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:36.625231 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:36.625249 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:36.625298 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:36.628924 (+ 3626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:36.629751 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:36.629764 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:36.629772 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:36.629928 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:36.630378 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:36.630383 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:36.630386 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:36.630881 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:36.631241 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:36.631569 (+ 328us) server_negotiation.cc:299] Negotiation successful
0807 18:14:36.631815 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 18:14:36.632855 24372 ts_tablet_manager.cc:1939] T 9f1c9890ceee42b985d560586e472fa7 P a98410440db74f4fb464af07dbb2b59a: Deleting consensus metadata
I20260807 18:14:36.633225 23949 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:14:36.634920 23677 catalog_manager.cc:4841] TS a98410440db74f4fb464af07dbb2b59a (127.17.110.3:35543): tablet 9f1c9890ceee42b985d560586e472fa7 (table test-table [id=08fa76a961ee479b8fac3b4b53e089ed]) successfully deleted
I20260807 18:14:36.728060 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 23788
I20260807 18:14:36.756259 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 23961
Aug 07 18:14:36 dist-test-slave-39l8 krb5kdc[23640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.2@KRBTEST.COM
I20260807 18:14:36.775146 24338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:36.757250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47470 (local address 127.17.110.2:35587)
0807 18:14:36.757477 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:36.757485 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:36.757509 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:14:36.757581 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:36.757586 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:36.757672 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:36.757811 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:36.759358 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.760509 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.761997 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.762629 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.762831 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:36.768283 (+ 5452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:36.768302 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:36.768320 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:36.768369 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:36.771905 (+ 3536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:36.772901 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:36.772909 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:36.772912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:36.773066 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:36.773519 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:36.773523 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:36.773525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:36.773963 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:36.774208 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:36.774555 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 18:14:36.774832 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 18:14:36.775876 24131 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:14:36.867579 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 23961
I20260807 18:14:36.899472 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 24146
Aug 07 18:14:36 dist-test-slave-39l8 krb5kdc[23640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.3@KRBTEST.COM
I20260807 18:14:36.917711 24332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:36.900485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37780 (local address 127.17.110.3:35543)
0807 18:14:36.900727 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:36.900739 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:36.900762 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:14:36.900846 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:36.900850 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:36.900915 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:36.901041 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:36.902401 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.903486 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.904804 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:36.905369 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:36.905529 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:36.910509 (+ 4980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:36.910527 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:36.910545 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:36.910606 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:36.914292 (+ 3686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:36.915246 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:36.915255 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:36.915258 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:36.915532 (+ 274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:36.915977 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:36.915983 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:36.915988 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:36.916385 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:36.916771 (+ 386us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:36.917049 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 18:14:36.917291 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 18:14:36.918581 24309 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:14:37.046450 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 24146
I20260807 18:14:37.074960 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 23655
I20260807 18:14:37.076210 23739 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:14:37.189965 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 23655
2026-08-07T18:14:37Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (5203 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T18:14:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:14:37Z Disabled control of system clock
I20260807 18:14:37.275278 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:43721
--webserver_interface=127.17.110.62
--webserver_port=0
--builtin_ntp_servers=127.17.110.20:43983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:43721
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 18:14:37.635926 24389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:37.636268 24389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:37.636358 24389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:37.644037 24389 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:14:37.644179 24389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:37.644218 24389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:37.644269 24389 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:14:37.644315 24389 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:14:37.655221 24389 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:43983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:43721
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:43721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:37.656569 24389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:37.658492 24389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:37.672679 24397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:37.672727 24398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:37.672689 24400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:37.673964 24389 server_base.cc:1034] running on GCE node
I20260807 18:14:37.675161 24389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:37.677232 24389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:37.678485 24389 hybrid_clock.cc:648] HybridClock initialized: now 1786126477678432 us; error 51 us; skew 500 ppm
I20260807 18:14:37.679003 24389 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:14:37.681785 24389 webserver.cc:460] Webserver started at http://127.17.110.62:38503/ using document root <none> and password file <none>
I20260807 18:14:37.682631 24389 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:37.682816 24389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:37.683199 24389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:37.686290 24389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "b1cd836a8401440bb0f851f0b0f6b6a9"
format_stamp: "Formatted at 2026-08-07 18:14:37 on dist-test-slave-39l8"
server_key: "fedc02808e29bea76285a0425636270e"
server_key_iv: "39093571dc56d4da4c727bc77c6c9b6c"
server_key_version: "encryptionkey@0"
I20260807 18:14:37.687197 24389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "b1cd836a8401440bb0f851f0b0f6b6a9"
format_stamp: "Formatted at 2026-08-07 18:14:37 on dist-test-slave-39l8"
server_key: "fedc02808e29bea76285a0425636270e"
server_key_iv: "39093571dc56d4da4c727bc77c6c9b6c"
server_key_version: "encryptionkey@0"
I20260807 18:14:37.693274 24389 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260807 18:14:37.697491 24408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:37.699179 24389 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 18:14:37.699395 24389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "b1cd836a8401440bb0f851f0b0f6b6a9"
format_stamp: "Formatted at 2026-08-07 18:14:37 on dist-test-slave-39l8"
server_key: "fedc02808e29bea76285a0425636270e"
server_key_iv: "39093571dc56d4da4c727bc77c6c9b6c"
server_key_version: "encryptionkey@0"
I20260807 18:14:37.699654 24389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:37.706153 24389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:37.707185 24389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:37.707543 24389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:37.728103 24389 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:43721
I20260807 18:14:37.728171 24481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:43721 every 8 connection(s)
I20260807 18:14:37.730037 24389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:14:37.731946 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 24389
I20260807 18:14:37.732223 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:14:37.733213 17848 external_mini_cluster.cc:1428] Setting key d4f628aaa403948d48af8a687c1c0d24
I20260807 18:14:37.736151 24482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:37.742940 24486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:37.735749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60616 (local address 127.17.110.62:43721)
0807 18:14:37.736932 (+ 1183us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:37.736945 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:37.736991 (+ 46us) server_negotiation.cc:407] Connection header received
0807 18:14:37.738842 (+ 1851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:37.738868 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:37.739329 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:37.739783 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:37.740968 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:37.740975 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:37.740987 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:14:37.741003 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:37.741090 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:14:37.741107 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:37.741373 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:37.741569 (+ 196us) server_negotiation.cc:299] Negotiation successful
0807 18:14:37.741868 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":691,"thread_start_us":448,"threads_started":1}
I20260807 18:14:37.750224 24482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1cd836a8401440bb0f851f0b0f6b6a9: Bootstrap starting.
I20260807 18:14:37.754400 24482 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1cd836a8401440bb0f851f0b0f6b6a9: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:37.755781 24482 log.cc:824] T 00000000000000000000000000000000 P b1cd836a8401440bb0f851f0b0f6b6a9: Log is configured to *not* fsync() on all Append() calls
I20260807 18:14:37.759516 24482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1cd836a8401440bb0f851f0b0f6b6a9: No bootstrap required, opened a new log
I20260807 18:14:37.768882 24482 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1cd836a8401440bb0f851f0b0f6b6a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1cd836a8401440bb0f851f0b0f6b6a9" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 43721 } }
I20260807 18:14:37.769352 24482 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1cd836a8401440bb0f851f0b0f6b6a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:37.769430 24482 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1cd836a8401440bb0f851f0b0f6b6a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1cd836a8401440bb0f851f0b0f6b6a9, State: Initialized, Role: FOLLOWER
I20260807 18:14:37.770117 24482 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1cd836a8401440bb0f851f0b0f6b6a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1cd836a8401440bb0f851f0b0f6b6a9" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 43721 } }
I20260807 18:14:37.770383 24482 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1cd836a8401440bb0f851f0b0f6b6a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:14:37.770498 24482 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1cd836a8401440bb0f851f0b0f6b6a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:14:37.770648 24482 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1cd836a8401440bb0f851f0b0f6b6a9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:37.773283 24482 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1cd836a8401440bb0f851f0b0f6b6a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1cd836a8401440bb0f851f0b0f6b6a9" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 43721 } }
I20260807 18:14:37.773869 24482 leader_election.cc:302] T 00000000000000000000000000000000 P b1cd836a8401440bb0f851f0b0f6b6a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1cd836a8401440bb0f851f0b0f6b6a9; no voters:
I20260807 18:14:37.774649 24482 leader_election.cc:288] T 00000000000000000000000000000000 P b1cd836a8401440bb0f851f0b0f6b6a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:14:37.774849 24488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1cd836a8401440bb0f851f0b0f6b6a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:14:37.776341 24488 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1cd836a8401440bb0f851f0b0f6b6a9 [term 1 LEADER]: Becoming Leader. State: Replica: b1cd836a8401440bb0f851f0b0f6b6a9, State: Running, Role: LEADER
I20260807 18:14:37.776961 24488 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1cd836a8401440bb0f851f0b0f6b6a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1cd836a8401440bb0f851f0b0f6b6a9" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 43721 } }
I20260807 18:14:37.777503 24482 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1cd836a8401440bb0f851f0b0f6b6a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:14:37.780686 24492 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1cd836a8401440bb0f851f0b0f6b6a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1cd836a8401440bb0f851f0b0f6b6a9. Latest consensus state: current_term: 1 leader_uuid: "b1cd836a8401440bb0f851f0b0f6b6a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1cd836a8401440bb0f851f0b0f6b6a9" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 43721 } } }
I20260807 18:14:37.781005 24491 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1cd836a8401440bb0f851f0b0f6b6a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1cd836a8401440bb0f851f0b0f6b6a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1cd836a8401440bb0f851f0b0f6b6a9" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 43721 } } }
I20260807 18:14:37.781605 24491 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1cd836a8401440bb0f851f0b0f6b6a9 [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:37.781616 24492 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1cd836a8401440bb0f851f0b0f6b6a9 [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:37.788625 24501 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:14:37.793599 24501 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:14:37.804919 24501 catalog_manager.cc:1347] Generated new cluster ID: 8cdea1e78e384a0b9c00b3b227914e75
I20260807 18:14:37.805256 24501 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:14:37.820257 24501 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:14:37.820773 24501 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:14:37.828953 24501 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1cd836a8401440bb0f851f0b0f6b6a9: Generated new TSK 0
I20260807 18:14:37.830029 24501 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:14:37.835602 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.1:0
--local_ip_for_outbound_sockets=127.17.110.1
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:43721
--builtin_ntp_servers=127.17.110.20:43983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 18:14:38.244379 24517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:38.244697 24517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:38.244874 24517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:38.252562 24517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:38.252699 24517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:38.252880 24517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.1
I20260807 18:14:38.262663 24517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:43983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:43721
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.1
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:38.264149 24517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:38.266104 24517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:38.280448 24524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:38.280414 24528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:38.280370 24526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:38.281317 24517 server_base.cc:1034] running on GCE node
I20260807 18:14:38.282048 24517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:38.283355 24517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:38.284642 24517 hybrid_clock.cc:648] HybridClock initialized: now 1786126478284569 us; error 76 us; skew 500 ppm
I20260807 18:14:38.285069 24517 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:14:38.287973 24517 webserver.cc:460] Webserver started at http://127.17.110.1:40875/ using document root <none> and password file <none>
I20260807 18:14:38.288889 24517 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:38.289052 24517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:38.289593 24517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:38.293190 24517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-0/data/instance:
uuid: "db96c40826094bc4992fee2f1e5896e3"
format_stamp: "Formatted at 2026-08-07 18:14:38 on dist-test-slave-39l8"
server_key: "843480f5892bdae959ab4ee8eb13daef"
server_key_iv: "c8780374d9b82336af3799fc244b3b1f"
server_key_version: "encryptionkey@0"
I20260807 18:14:38.294225 24517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance:
uuid: "db96c40826094bc4992fee2f1e5896e3"
format_stamp: "Formatted at 2026-08-07 18:14:38 on dist-test-slave-39l8"
server_key: "843480f5892bdae959ab4ee8eb13daef"
server_key_iv: "c8780374d9b82336af3799fc244b3b1f"
server_key_version: "encryptionkey@0"
I20260807 18:14:38.300488 24517 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 18:14:38.304690 24534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:38.306223 24517 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:14:38.306464 24517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-0/wal
uuid: "db96c40826094bc4992fee2f1e5896e3"
format_stamp: "Formatted at 2026-08-07 18:14:38 on dist-test-slave-39l8"
server_key: "843480f5892bdae959ab4ee8eb13daef"
server_key_iv: "c8780374d9b82336af3799fc244b3b1f"
server_key_version: "encryptionkey@0"
I20260807 18:14:38.306738 24517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:38.312906 24517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:38.313848 24517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:38.314163 24517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:38.315282 24517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:14:38.316865 24517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:14:38.316994 24517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:38.317142 24517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:14:38.317232 24517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:38.353662 24517 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.1:37311
I20260807 18:14:38.353729 24686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.1:37311 every 8 connection(s)
I20260807 18:14:38.355206 24517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
I20260807 18:14:38.357803 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 24517
I20260807 18:14:38.358044 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance
I20260807 18:14:38.358774 17848 external_mini_cluster.cc:1428] Setting key ae1eaadfa301f0c3738164c2c139f0c5
I20260807 18:14:38.362406 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.2:0
--local_ip_for_outbound_sockets=127.17.110.2
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:43721
--builtin_ntp_servers=127.17.110.20:43983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 18:14:38.365275 24691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:38.359687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:52693 (local address 127.17.110.62:43721)
0807 18:14:38.360150 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:38.360154 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:38.360174 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:14:38.362487 (+ 2313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:38.362492 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:38.362726 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:38.362845 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:38.364531 (+ 1686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:38.364536 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:38.364539 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:14:38.364557 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:38.364605 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:14:38.364618 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:38.364743 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:38.364969 (+ 226us) server_negotiation.cc:299] Negotiation successful
0807 18:14:38.365027 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":242,"threads_started":1}
I20260807 18:14:38.366845 24690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:38.358609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:43721 (local address 127.17.110.1:52693)
0807 18:14:38.359953 (+ 1344us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:38.360019 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:38.362113 (+ 2094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:38.363152 (+ 1039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:38.363167 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:38.363323 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:38.364119 (+ 796us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:14:38.364126 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:38.364187 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:14:38.364332 (+ 145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:38.364726 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:38.364739 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:38.364749 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 18:14:38.365057 (+ 308us) client_negotiation.cc:240] Negotiation successful
0807 18:14:38.365389 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":350,"threads_started":1}
I20260807 18:14:38.368683 24687 heartbeater.cc:347] Connected to a master server at 127.17.110.62:43721
I20260807 18:14:38.369099 24687 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:38.369831 24687 heartbeater.cc:511] Master 127.17.110.62:43721 requested a full tablet report, sending...
I20260807 18:14:38.372501 24431 ts_manager.cc:194] Registered new tserver with Master: db96c40826094bc4992fee2f1e5896e3 (127.17.110.1:37311)
W20260807 18:14:38.738875 24692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:38.739291 24692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:38.739421 24692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:38.751399 24692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:38.751545 24692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:38.751715 24692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.2
I20260807 18:14:38.761716 24692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:43983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:43721
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.2
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:38.763228 24692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:38.765234 24692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:38.779627 24698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:38.779548 24699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:38.779601 24705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:38.780536 24692 server_base.cc:1034] running on GCE node
I20260807 18:14:38.781196 24692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:38.782207 24692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:38.783514 24692 hybrid_clock.cc:648] HybridClock initialized: now 1786126478783444 us; error 57 us; skew 500 ppm
I20260807 18:14:38.783902 24692 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:14:38.786348 24692 webserver.cc:460] Webserver started at http://127.17.110.2:39267/ using document root <none> and password file <none>
I20260807 18:14:38.787209 24692 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:38.787340 24692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:38.787712 24692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:38.790875 24692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-1/data/instance:
uuid: "f876fb6e84c14537ada2e2998e3bc681"
format_stamp: "Formatted at 2026-08-07 18:14:38 on dist-test-slave-39l8"
server_key: "ae0ae73142da1cd4bb40b4f314297555"
server_key_iv: "01163b77c18933e009f2c4222e727df4"
server_key_version: "encryptionkey@0"
I20260807 18:14:38.791788 24692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance:
uuid: "f876fb6e84c14537ada2e2998e3bc681"
format_stamp: "Formatted at 2026-08-07 18:14:38 on dist-test-slave-39l8"
server_key: "ae0ae73142da1cd4bb40b4f314297555"
server_key_iv: "01163b77c18933e009f2c4222e727df4"
server_key_version: "encryptionkey@0"
I20260807 18:14:38.797777 24692 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 18:14:38.801990 24713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:38.803946 24692 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:14:38.804211 24692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-1/wal
uuid: "f876fb6e84c14537ada2e2998e3bc681"
format_stamp: "Formatted at 2026-08-07 18:14:38 on dist-test-slave-39l8"
server_key: "ae0ae73142da1cd4bb40b4f314297555"
server_key_iv: "01163b77c18933e009f2c4222e727df4"
server_key_version: "encryptionkey@0"
I20260807 18:14:38.804483 24692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:38.810830 24692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:38.811703 24692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:38.812019 24692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:38.812963 24692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:14:38.814437 24692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:14:38.814530 24692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:38.814617 24692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:14:38.814673 24692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:38.850106 24692 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.2:42243
I20260807 18:14:38.850155 24876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.2:42243 every 8 connection(s)
I20260807 18:14:38.851557 24692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
I20260807 18:14:38.854100 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 24692
I20260807 18:14:38.854331 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance
I20260807 18:14:38.854975 17848 external_mini_cluster.cc:1428] Setting key 8420cd1b68f036fe916a9ed93e035f7f
I20260807 18:14:38.858502 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.3:0
--local_ip_for_outbound_sockets=127.17.110.3
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:43721
--builtin_ntp_servers=127.17.110.20:43983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 18:14:38.861609 24691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:38.854531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:36899 (local address 127.17.110.62:43721)
0807 18:14:38.854789 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:38.854797 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:38.856600 (+ 1803us) server_negotiation.cc:407] Connection header received
0807 18:14:38.859008 (+ 2408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:38.859012 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:38.859068 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:38.859162 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:38.860874 (+ 1712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:38.860879 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:38.860882 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:14:38.860900 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:38.860937 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:14:38.860950 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:38.861075 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:38.861336 (+ 261us) server_negotiation.cc:299] Negotiation successful
0807 18:14:38.861398 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 18:14:38.863101 24881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:38.855257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:43721 (local address 127.17.110.2:36899)
0807 18:14:38.856439 (+ 1182us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:38.856478 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:38.858635 (+ 2157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:38.859475 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:38.859491 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:38.859637 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:38.860434 (+ 797us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:14:38.860440 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:38.860508 (+ 68us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:14:38.860640 (+ 132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:38.861068 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:38.861083 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:38.861095 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 18:14:38.861402 (+ 307us) client_negotiation.cc:240] Negotiation successful
0807 18:14:38.861700 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":356,"threads_started":1}
I20260807 18:14:38.864729 24879 heartbeater.cc:347] Connected to a master server at 127.17.110.62:43721
I20260807 18:14:38.865078 24879 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:38.865736 24879 heartbeater.cc:511] Master 127.17.110.62:43721 requested a full tablet report, sending...
I20260807 18:14:38.867297 24431 ts_manager.cc:194] Registered new tserver with Master: f876fb6e84c14537ada2e2998e3bc681 (127.17.110.2:42243)
W20260807 18:14:39.216466 24882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:39.216845 24882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:39.217015 24882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:39.224936 24882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:39.225098 24882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:39.225296 24882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.3
I20260807 18:14:39.235122 24882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:43983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:43721
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.3
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:39.236582 24882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:39.238551 24882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:39.253856 24890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:39.253993 24889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:39.253993 24892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:39.255192 24882 server_base.cc:1034] running on GCE node
I20260807 18:14:39.255875 24882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:39.256924 24882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:39.258163 24882 hybrid_clock.cc:648] HybridClock initialized: now 1786126479258100 us; error 51 us; skew 500 ppm
I20260807 18:14:39.258564 24882 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:14:39.266582 24882 webserver.cc:460] Webserver started at http://127.17.110.3:41025/ using document root <none> and password file <none>
I20260807 18:14:39.267485 24882 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:39.267625 24882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:39.268018 24882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:39.271342 24882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-2/data/instance:
uuid: "7b60df1008ed476f85387868a8aed4d6"
format_stamp: "Formatted at 2026-08-07 18:14:39 on dist-test-slave-39l8"
server_key: "bd2fc206793f56f79948a347ee024e31"
server_key_iv: "5946673e322b063a9f9c717e4de6a760"
server_key_version: "encryptionkey@0"
I20260807 18:14:39.272276 24882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance:
uuid: "7b60df1008ed476f85387868a8aed4d6"
format_stamp: "Formatted at 2026-08-07 18:14:39 on dist-test-slave-39l8"
server_key: "bd2fc206793f56f79948a347ee024e31"
server_key_iv: "5946673e322b063a9f9c717e4de6a760"
server_key_version: "encryptionkey@0"
I20260807 18:14:39.278512 24882 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.003s
I20260807 18:14:39.282845 24900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:39.284492 24882 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:14:39.284705 24882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-2/wal
uuid: "7b60df1008ed476f85387868a8aed4d6"
format_stamp: "Formatted at 2026-08-07 18:14:39 on dist-test-slave-39l8"
server_key: "bd2fc206793f56f79948a347ee024e31"
server_key_iv: "5946673e322b063a9f9c717e4de6a760"
server_key_version: "encryptionkey@0"
I20260807 18:14:39.284978 24882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:39.291596 24882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:39.292578 24882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:39.292900 24882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:39.293993 24882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:14:39.295791 24882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:14:39.295923 24882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:39.296066 24882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:14:39.296156 24882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:39.332335 24882 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.3:44669
I20260807 18:14:39.332394 25047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.3:44669 every 8 connection(s)
I20260807 18:14:39.333781 24882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
I20260807 18:14:39.336884 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 24882
I20260807 18:14:39.337082 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance
I20260807 18:14:39.337553 17848 external_mini_cluster.cc:1428] Setting key 9705e82c53157cddb362896dc428641b
I20260807 18:14:39.342890 24691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:39.336763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:59383 (local address 127.17.110.62:43721)
0807 18:14:39.337018 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:39.337025 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:39.338613 (+ 1588us) server_negotiation.cc:407] Connection header received
0807 18:14:39.340422 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:39.340428 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:39.340499 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:39.340632 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:39.341979 (+ 1347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:39.341986 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:39.341990 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:14:39.342011 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:39.342053 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:14:39.342065 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:39.342190 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:39.342465 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 18:14:39.342538 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 18:14:39.343894 25053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:39.337312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:43721 (local address 127.17.110.3:59383)
0807 18:14:39.338436 (+ 1124us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:39.338484 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:39.340133 (+ 1649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:39.340885 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:39.340894 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:39.340991 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:39.341657 (+ 666us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:14:39.341659 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:39.341701 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:14:39.341796 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:39.342180 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:39.342191 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:39.342200 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 18:14:39.342554 (+ 354us) client_negotiation.cc:240] Negotiation successful
0807 18:14:39.342871 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":357,"threads_started":1}
I20260807 18:14:39.345396 25049 heartbeater.cc:347] Connected to a master server at 127.17.110.62:43721
I20260807 18:14:39.345750 25049 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:39.346431 25049 heartbeater.cc:511] Master 127.17.110.62:43721 requested a full tablet report, sending...
I20260807 18:14:39.348096 24431 ts_manager.cc:194] Registered new tserver with Master: 7b60df1008ed476f85387868a8aed4d6 (127.17.110.3:44669)
I20260807 18:14:39.353237 17848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:14:39.361337 24691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:39.358559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60622 (local address 127.17.110.62:43721)
0807 18:14:39.358830 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:39.358835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:39.359371 (+ 536us) server_negotiation.cc:407] Connection header received
0807 18:14:39.359515 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:39.359520 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:39.359606 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:39.359707 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:39.360763 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:39.360788 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:39.360791 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:14:39.360808 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:39.360843 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:14:39.360855 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:39.360968 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:39.361067 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 18:14:39.361120 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:14:39.366159 24431 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60622:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:14:39.373500 24431 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:14:39.376506 24687 heartbeater.cc:503] Master 127.17.110.62:43721 was elected leader, sending a full tablet report...
I20260807 18:14:39.404886 25071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:39.388529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48182 (local address 127.17.110.3:44669)
0807 18:14:39.401546 (+ 13017us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:39.401567 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:39.401611 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:14:39.401703 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:39.401712 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:39.402004 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:39.402216 (+ 212us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:39.403912 (+ 1696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:39.403919 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:39.403931 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:14:39.403946 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:39.403990 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:14:39.404003 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:39.404231 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:39.404322 (+ 91us) server_negotiation.cc:299] Negotiation successful
0807 18:14:39.404432 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12823,"thread_start_us":12628,"threads_started":1}
I20260807 18:14:39.405048 25072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:39.387776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:44669 (local address 127.0.0.1:48182)
0807 18:14:39.399800 (+ 12024us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:39.399829 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:39.399934 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:39.402239 (+ 2305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:39.402248 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:39.402367 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:39.403397 (+ 1030us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:14:39.403403 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:39.403439 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:14:39.403586 (+ 147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:39.404118 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:39.404132 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:39.404141 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 18:14:39.404335 (+ 194us) client_negotiation.cc:240] Negotiation successful
0807 18:14:39.404432 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11886,"thread_start_us":1386,"threads_started":1}
I20260807 18:14:39.409780 25067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:39.387757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58626 (local address 127.17.110.2:42243)
0807 18:14:39.406266 (+ 18509us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:39.406275 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:39.406306 (+ 31us) server_negotiation.cc:407] Connection header received
0807 18:14:39.406377 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:39.406385 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:39.406653 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:39.407862 (+ 1209us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:39.408677 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:39.408687 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:39.408722 (+ 35us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:14:39.408753 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:39.408820 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:14:39.408925 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:39.409150 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:39.409302 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 18:14:39.409416 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18340,"thread_start_us":18209,"threads_started":1}
I20260807 18:14:39.409860 25068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:39.387736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:42243 (local address 127.0.0.1:58626)
0807 18:14:39.388559 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:39.388614 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:39.388818 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:39.406877 (+ 18059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:39.406883 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:39.406911 (+ 28us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:39.407722 (+ 811us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:14:39.407724 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:39.407740 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:14:39.407781 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:39.409092 (+ 1311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:39.409098 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:39.409104 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 18:14:39.409266 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 18:14:39.409372 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":17527,"threads_started":1}
I20260807 18:14:39.409734 24969 tablet_service.cc:1467] Processing CreateTablet for tablet 5c51c8593288459a8135b6fc1dc08841 (DEFAULT_TABLE table=test-table [id=57b0e6900a084d5a999127200b8c1624]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:14:39.411562 24969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c51c8593288459a8135b6fc1dc08841. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:39.411681 25069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:39.389932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:37311 (local address 127.0.0.1:34462)
0807 18:14:39.405571 (+ 15639us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:39.405602 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:39.405735 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:39.408075 (+ 2340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:39.408080 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:39.408106 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:39.408852 (+ 746us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:14:39.408855 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:39.408880 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:14:39.408929 (+ 49us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:39.410802 (+ 1873us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:39.410807 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:39.410814 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 18:14:39.410984 (+ 170us) client_negotiation.cc:240] Negotiation successful
0807 18:14:39.411048 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15508,"thread_start_us":9545,"threads_started":1}
I20260807 18:14:39.412448 25073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:39.390584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34462 (local address 127.17.110.1:37311)
0807 18:14:39.400209 (+ 9625us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:39.400222 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:39.405732 (+ 5510us) server_negotiation.cc:407] Connection header received
0807 18:14:39.406426 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:39.406442 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:39.406676 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:39.406891 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:39.410233 (+ 3342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:39.410246 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:39.410265 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:14:39.410363 (+ 98us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:39.410444 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:14:39.410471 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:39.410727 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:39.410986 (+ 259us) server_negotiation.cc:299] Negotiation successful
0807 18:14:39.411099 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9318,"thread_start_us":9407,"threads_started":1}
I20260807 18:14:39.414358 24777 tablet_service.cc:1467] Processing CreateTablet for tablet 5c51c8593288459a8135b6fc1dc08841 (DEFAULT_TABLE table=test-table [id=57b0e6900a084d5a999127200b8c1624]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:14:39.415999 24777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c51c8593288459a8135b6fc1dc08841. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:39.416476 24600 tablet_service.cc:1467] Processing CreateTablet for tablet 5c51c8593288459a8135b6fc1dc08841 (DEFAULT_TABLE table=test-table [id=57b0e6900a084d5a999127200b8c1624]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:14:39.418015 24600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c51c8593288459a8135b6fc1dc08841. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:39.427646 25074 tablet_bootstrap.cc:492] T 5c51c8593288459a8135b6fc1dc08841 P 7b60df1008ed476f85387868a8aed4d6: Bootstrap starting.
I20260807 18:14:39.432463 25074 tablet_bootstrap.cc:654] T 5c51c8593288459a8135b6fc1dc08841 P 7b60df1008ed476f85387868a8aed4d6: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:39.433725 25074 log.cc:824] T 5c51c8593288459a8135b6fc1dc08841 P 7b60df1008ed476f85387868a8aed4d6: Log is configured to *not* fsync() on all Append() calls
I20260807 18:14:39.435842 25075 tablet_bootstrap.cc:492] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681: Bootstrap starting.
I20260807 18:14:39.437070 25077 tablet_bootstrap.cc:492] T 5c51c8593288459a8135b6fc1dc08841 P db96c40826094bc4992fee2f1e5896e3: Bootstrap starting.
I20260807 18:14:39.440724 25075 tablet_bootstrap.cc:654] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:39.442157 25075 log.cc:824] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681: Log is configured to *not* fsync() on all Append() calls
I20260807 18:14:39.442517 25077 tablet_bootstrap.cc:654] T 5c51c8593288459a8135b6fc1dc08841 P db96c40826094bc4992fee2f1e5896e3: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:39.444157 25077 log.cc:824] T 5c51c8593288459a8135b6fc1dc08841 P db96c40826094bc4992fee2f1e5896e3: Log is configured to *not* fsync() on all Append() calls
I20260807 18:14:39.445905 25075 tablet_bootstrap.cc:492] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681: No bootstrap required, opened a new log
I20260807 18:14:39.445916 25074 tablet_bootstrap.cc:492] T 5c51c8593288459a8135b6fc1dc08841 P 7b60df1008ed476f85387868a8aed4d6: No bootstrap required, opened a new log
I20260807 18:14:39.446210 25075 ts_tablet_manager.cc:1397] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260807 18:14:39.446261 25074 ts_tablet_manager.cc:1397] T 5c51c8593288459a8135b6fc1dc08841 P 7b60df1008ed476f85387868a8aed4d6: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.008s
I20260807 18:14:39.447907 25077 tablet_bootstrap.cc:492] T 5c51c8593288459a8135b6fc1dc08841 P db96c40826094bc4992fee2f1e5896e3: No bootstrap required, opened a new log
I20260807 18:14:39.448259 25077 ts_tablet_manager.cc:1397] T 5c51c8593288459a8135b6fc1dc08841 P db96c40826094bc4992fee2f1e5896e3: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260807 18:14:39.455595 25075 raft_consensus.cc:348] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f876fb6e84c14537ada2e2998e3bc681" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 42243 } } peers { permanent_uuid: "7b60df1008ed476f85387868a8aed4d6" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44669 } } peers { permanent_uuid: "db96c40826094bc4992fee2f1e5896e3" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37311 } }
I20260807 18:14:39.456236 25075 raft_consensus.cc:374] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:39.456353 25075 raft_consensus.cc:729] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f876fb6e84c14537ada2e2998e3bc681, State: Initialized, Role: FOLLOWER
I20260807 18:14:39.457122 25075 consensus_queue.cc:260] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f876fb6e84c14537ada2e2998e3bc681" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 42243 } } peers { permanent_uuid: "7b60df1008ed476f85387868a8aed4d6" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44669 } } peers { permanent_uuid: "db96c40826094bc4992fee2f1e5896e3" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37311 } }
I20260807 18:14:39.458868 24879 heartbeater.cc:503] Master 127.17.110.62:43721 was elected leader, sending a full tablet report...
I20260807 18:14:39.459734 25074 raft_consensus.cc:348] T 5c51c8593288459a8135b6fc1dc08841 P 7b60df1008ed476f85387868a8aed4d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f876fb6e84c14537ada2e2998e3bc681" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 42243 } } peers { permanent_uuid: "7b60df1008ed476f85387868a8aed4d6" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44669 } } peers { permanent_uuid: "db96c40826094bc4992fee2f1e5896e3" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37311 } }
I20260807 18:14:39.460354 25074 raft_consensus.cc:374] T 5c51c8593288459a8135b6fc1dc08841 P 7b60df1008ed476f85387868a8aed4d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:39.460493 25074 raft_consensus.cc:729] T 5c51c8593288459a8135b6fc1dc08841 P 7b60df1008ed476f85387868a8aed4d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b60df1008ed476f85387868a8aed4d6, State: Initialized, Role: FOLLOWER
I20260807 18:14:39.460966 25077 raft_consensus.cc:348] T 5c51c8593288459a8135b6fc1dc08841 P db96c40826094bc4992fee2f1e5896e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f876fb6e84c14537ada2e2998e3bc681" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 42243 } } peers { permanent_uuid: "7b60df1008ed476f85387868a8aed4d6" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44669 } } peers { permanent_uuid: "db96c40826094bc4992fee2f1e5896e3" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37311 } }
I20260807 18:14:39.461508 25077 raft_consensus.cc:374] T 5c51c8593288459a8135b6fc1dc08841 P db96c40826094bc4992fee2f1e5896e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:39.461637 25077 raft_consensus.cc:729] T 5c51c8593288459a8135b6fc1dc08841 P db96c40826094bc4992fee2f1e5896e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db96c40826094bc4992fee2f1e5896e3, State: Initialized, Role: FOLLOWER
I20260807 18:14:39.461436 25074 consensus_queue.cc:260] T 5c51c8593288459a8135b6fc1dc08841 P 7b60df1008ed476f85387868a8aed4d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f876fb6e84c14537ada2e2998e3bc681" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 42243 } } peers { permanent_uuid: "7b60df1008ed476f85387868a8aed4d6" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44669 } } peers { permanent_uuid: "db96c40826094bc4992fee2f1e5896e3" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37311 } }
I20260807 18:14:39.462007 25075 ts_tablet_manager.cc:1428] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681: Time spent starting tablet: real 0.016s user 0.010s sys 0.006s
I20260807 18:14:39.462456 25077 consensus_queue.cc:260] T 5c51c8593288459a8135b6fc1dc08841 P db96c40826094bc4992fee2f1e5896e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f876fb6e84c14537ada2e2998e3bc681" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 42243 } } peers { permanent_uuid: "7b60df1008ed476f85387868a8aed4d6" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44669 } } peers { permanent_uuid: "db96c40826094bc4992fee2f1e5896e3" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37311 } }
I20260807 18:14:39.463301 25049 heartbeater.cc:503] Master 127.17.110.62:43721 was elected leader, sending a full tablet report...
I20260807 18:14:39.464076 25074 ts_tablet_manager.cc:1428] T 5c51c8593288459a8135b6fc1dc08841 P 7b60df1008ed476f85387868a8aed4d6: Time spent starting tablet: real 0.018s user 0.019s sys 0.001s
I20260807 18:14:39.466312 25077 ts_tablet_manager.cc:1428] T 5c51c8593288459a8135b6fc1dc08841 P db96c40826094bc4992fee2f1e5896e3: Time spent starting tablet: real 0.018s user 0.009s sys 0.008s
I20260807 18:14:39.542025 25085 raft_consensus.cc:482] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:14:39.542372 25085 raft_consensus.cc:504] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f876fb6e84c14537ada2e2998e3bc681" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 42243 } } peers { permanent_uuid: "7b60df1008ed476f85387868a8aed4d6" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44669 } } peers { permanent_uuid: "db96c40826094bc4992fee2f1e5896e3" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37311 } }
I20260807 18:14:39.544317 25085 leader_election.cc:288] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b60df1008ed476f85387868a8aed4d6 (127.17.110.3:44669), db96c40826094bc4992fee2f1e5896e3 (127.17.110.1:37311)
I20260807 18:14:39.548622 25071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:39.544982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:59259 (local address 127.17.110.3:44669)
0807 18:14:39.545253 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:39.545260 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:39.545581 (+ 321us) server_negotiation.cc:407] Connection header received
0807 18:14:39.545872 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:39.545877 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:39.545946 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:39.546055 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:39.547662 (+ 1607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:39.547671 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:39.547675 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:14:39.547703 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:39.547754 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:14:39.547771 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:39.547932 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:39.548232 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 18:14:39.548331 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260807 18:14:39.548959 25091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:39.544679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:44669 (local address 127.17.110.2:59259)
0807 18:14:39.545437 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:39.545456 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:39.545639 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:39.546288 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:39.546295 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:39.546332 (+ 37us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:39.547307 (+ 975us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:14:39.547311 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:39.547332 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:14:39.547390 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:39.548017 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:39.548023 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:39.548028 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 18:14:39.548238 (+ 210us) client_negotiation.cc:240] Negotiation successful
0807 18:14:39.548308 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":441,"threads_started":1}
I20260807 18:14:39.549950 25073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:39.545767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:42519 (local address 127.17.110.1:37311)
0807 18:14:39.546007 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:39.546015 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:39.546795 (+ 780us) server_negotiation.cc:407] Connection header received
0807 18:14:39.547065 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:39.547071 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:39.547173 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:39.547306 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:39.548965 (+ 1659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:39.548971 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:39.548974 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:14:39.548996 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:39.549037 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:14:39.549124 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:39.549251 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:39.549583 (+ 332us) server_negotiation.cc:299] Negotiation successful
0807 18:14:39.549659 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 18:14:39.550184 25092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:39.545607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:37311 (local address 127.17.110.2:42519)
0807 18:14:39.546626 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:39.546646 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:39.546846 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:39.547364 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:39.547369 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:39.547395 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:39.548470 (+ 1075us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:14:39.548474 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:39.548492 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:14:39.548614 (+ 122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:39.549400 (+ 786us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:39.549404 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:39.549407 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 18:14:39.549520 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 18:14:39.549623 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":895,"thread_start_us":618,"threads_started":1}
I20260807 18:14:39.550490 24993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c51c8593288459a8135b6fc1dc08841" candidate_uuid: "f876fb6e84c14537ada2e2998e3bc681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b60df1008ed476f85387868a8aed4d6" is_pre_election: true
I20260807 18:14:39.551180 24993 raft_consensus.cc:2454] T 5c51c8593288459a8135b6fc1dc08841 P 7b60df1008ed476f85387868a8aed4d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f876fb6e84c14537ada2e2998e3bc681 in term 0.
I20260807 18:14:39.551520 24626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c51c8593288459a8135b6fc1dc08841" candidate_uuid: "f876fb6e84c14537ada2e2998e3bc681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db96c40826094bc4992fee2f1e5896e3" is_pre_election: true
I20260807 18:14:39.552026 24718 leader_election.cc:302] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b60df1008ed476f85387868a8aed4d6, f876fb6e84c14537ada2e2998e3bc681; no voters:
I20260807 18:14:39.552436 24626 raft_consensus.cc:2454] T 5c51c8593288459a8135b6fc1dc08841 P db96c40826094bc4992fee2f1e5896e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f876fb6e84c14537ada2e2998e3bc681 in term 0.
I20260807 18:14:39.552482 25085 raft_consensus.cc:2781] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:14:39.552676 25085 raft_consensus.cc:482] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:14:39.552935 25085 raft_consensus.cc:3037] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:39.555984 25085 raft_consensus.cc:504] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f876fb6e84c14537ada2e2998e3bc681" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 42243 } } peers { permanent_uuid: "7b60df1008ed476f85387868a8aed4d6" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44669 } } peers { permanent_uuid: "db96c40826094bc4992fee2f1e5896e3" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37311 } }
I20260807 18:14:39.556900 25085 leader_election.cc:288] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681 [CANDIDATE]: Term 1 election: Requested vote from peers 7b60df1008ed476f85387868a8aed4d6 (127.17.110.3:44669), db96c40826094bc4992fee2f1e5896e3 (127.17.110.1:37311)
I20260807 18:14:39.557762 24626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c51c8593288459a8135b6fc1dc08841" candidate_uuid: "f876fb6e84c14537ada2e2998e3bc681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db96c40826094bc4992fee2f1e5896e3"
I20260807 18:14:39.558029 24626 raft_consensus.cc:3037] T 5c51c8593288459a8135b6fc1dc08841 P db96c40826094bc4992fee2f1e5896e3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:39.557960 24993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c51c8593288459a8135b6fc1dc08841" candidate_uuid: "f876fb6e84c14537ada2e2998e3bc681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b60df1008ed476f85387868a8aed4d6"
I20260807 18:14:39.558360 24993 raft_consensus.cc:3037] T 5c51c8593288459a8135b6fc1dc08841 P 7b60df1008ed476f85387868a8aed4d6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:39.560752 24626 raft_consensus.cc:2454] T 5c51c8593288459a8135b6fc1dc08841 P db96c40826094bc4992fee2f1e5896e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f876fb6e84c14537ada2e2998e3bc681 in term 1.
I20260807 18:14:39.561071 24993 raft_consensus.cc:2454] T 5c51c8593288459a8135b6fc1dc08841 P 7b60df1008ed476f85387868a8aed4d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f876fb6e84c14537ada2e2998e3bc681 in term 1.
I20260807 18:14:39.561497 24718 leader_election.cc:302] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db96c40826094bc4992fee2f1e5896e3, f876fb6e84c14537ada2e2998e3bc681; no voters:
I20260807 18:14:39.561936 25085 raft_consensus.cc:2781] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:14:39.562503 25085 raft_consensus.cc:686] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681 [term 1 LEADER]: Becoming Leader. State: Replica: f876fb6e84c14537ada2e2998e3bc681, State: Running, Role: LEADER
I20260807 18:14:39.563133 25085 consensus_queue.cc:237] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f876fb6e84c14537ada2e2998e3bc681" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 42243 } } peers { permanent_uuid: "7b60df1008ed476f85387868a8aed4d6" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44669 } } peers { permanent_uuid: "db96c40826094bc4992fee2f1e5896e3" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37311 } }
I20260807 18:14:39.569104 24431 catalog_manager.cc:5495] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681 reported cstate change: term changed from 0 to 1, leader changed from <none> to f876fb6e84c14537ada2e2998e3bc681 (127.17.110.2). New cstate: current_term: 1 leader_uuid: "f876fb6e84c14537ada2e2998e3bc681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f876fb6e84c14537ada2e2998e3bc681" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 42243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b60df1008ed476f85387868a8aed4d6" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db96c40826094bc4992fee2f1e5896e3" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37311 } health_report { overall_health: UNKNOWN } } }
W20260807 18:14:39.585816 25050 tablet.cc:2369] T 5c51c8593288459a8135b6fc1dc08841 P 7b60df1008ed476f85387868a8aed4d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:14:39.599627 25067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:39.596390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58630 (local address 127.17.110.2:42243)
0807 18:14:39.596796 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:39.596803 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:39.596934 (+ 131us) server_negotiation.cc:407] Connection header received
0807 18:14:39.597277 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:39.597281 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:39.597381 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:39.597517 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:39.598853 (+ 1336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:39.598860 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:39.598864 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:14:39.598889 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:39.598935 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:14:39.598950 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:39.599099 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:39.599235 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 18:14:39.599313 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239}
W20260807 18:14:39.603842 24880 tablet.cc:2369] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:14:39.607901 24689 tablet.cc:2369] T 5c51c8593288459a8135b6fc1dc08841 P db96c40826094bc4992fee2f1e5896e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:14:39.610173 24993 raft_consensus.cc:1261] T 5c51c8593288459a8135b6fc1dc08841 P 7b60df1008ed476f85387868a8aed4d6 [term 1 FOLLOWER]: Refusing update from remote peer f876fb6e84c14537ada2e2998e3bc681: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:14:39.610173 24626 raft_consensus.cc:1261] T 5c51c8593288459a8135b6fc1dc08841 P db96c40826094bc4992fee2f1e5896e3 [term 1 FOLLOWER]: Refusing update from remote peer f876fb6e84c14537ada2e2998e3bc681: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:14:39.611410 25093 consensus_queue.cc:1035] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db96c40826094bc4992fee2f1e5896e3" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:14:39.611838 25085 consensus_queue.cc:1035] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b60df1008ed476f85387868a8aed4d6" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:14:39.644171 24429 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:14:39.644578 24429 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:14:39.650355 24429 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b1cd836a8401440bb0f851f0b0f6b6a9: Sending DeleteTablet for 3 replicas of tablet 5c51c8593288459a8135b6fc1dc08841
I20260807 18:14:39.652048 24777 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c51c8593288459a8135b6fc1dc08841 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:14:39 UTC) from {username='slave'} at 127.0.0.1:58626
I20260807 18:14:39.653273 25111 tablet_replica.cc:330] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681: stopping tablet replica
I20260807 18:14:39.653954 25111 raft_consensus.cc:2229] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:14:39.653968 24600 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c51c8593288459a8135b6fc1dc08841 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:14:39 UTC) from {username='slave'} at 127.0.0.1:34462
I20260807 18:14:39.654942 25111 raft_consensus.cc:2258] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:14:39.656414 24969 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c51c8593288459a8135b6fc1dc08841 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:14:39 UTC) from {username='slave'} at 127.0.0.1:48182
I20260807 18:14:39.659014 25111 ts_tablet_manager.cc:1905] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:14:39.660467 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 24517
I20260807 18:14:39.664172 25113 tablet_replica.cc:330] T 5c51c8593288459a8135b6fc1dc08841 P 7b60df1008ed476f85387868a8aed4d6: stopping tablet replica
I20260807 18:14:39.664754 25113 raft_consensus.cc:2229] T 5c51c8593288459a8135b6fc1dc08841 P 7b60df1008ed476f85387868a8aed4d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:14:39.668269 25112 tablet_replica.cc:330] T 5c51c8593288459a8135b6fc1dc08841 P db96c40826094bc4992fee2f1e5896e3: stopping tablet replica
I20260807 18:14:39.668778 25112 raft_consensus.cc:2229] T 5c51c8593288459a8135b6fc1dc08841 P db96c40826094bc4992fee2f1e5896e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:14:39.669365 25112 raft_consensus.cc:2258] T 5c51c8593288459a8135b6fc1dc08841 P db96c40826094bc4992fee2f1e5896e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:14:39.669713 25073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:39.665119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34466 (local address 127.17.110.1:37311)
0807 18:14:39.665357 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:39.665364 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:39.665386 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:14:39.665457 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:39.665463 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:39.665585 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:39.665703 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:39.668706 (+ 3003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:39.668714 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:39.668720 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:14:39.668746 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:39.668799 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:14:39.668822 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:39.668965 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:39.669259 (+ 294us) server_negotiation.cc:299] Negotiation successful
0807 18:14:39.669335 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260807 18:14:39.665417 25113 raft_consensus.cc:2258] T 5c51c8593288459a8135b6fc1dc08841 P 7b60df1008ed476f85387868a8aed4d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:14:39.670984 24679 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:14:39.673522 25113 ts_tablet_manager.cc:1905] T 5c51c8593288459a8135b6fc1dc08841 P 7b60df1008ed476f85387868a8aed4d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:14:39.676594 25111 ts_tablet_manager.cc:1918] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:14:39.677095 25111 log.cc:1196] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-1/wal/wals/5c51c8593288459a8135b6fc1dc08841
I20260807 18:14:39.677908 25111 ts_tablet_manager.cc:1939] T 5c51c8593288459a8135b6fc1dc08841 P f876fb6e84c14537ada2e2998e3bc681: Deleting consensus metadata
I20260807 18:14:39.680841 24410 catalog_manager.cc:4841] TS f876fb6e84c14537ada2e2998e3bc681 (127.17.110.2:42243): tablet 5c51c8593288459a8135b6fc1dc08841 (table test-table [id=57b0e6900a084d5a999127200b8c1624]) successfully deleted
I20260807 18:14:39.684635 25113 ts_tablet_manager.cc:1918] T 5c51c8593288459a8135b6fc1dc08841 P 7b60df1008ed476f85387868a8aed4d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:14:39.684921 25113 log.cc:1196] T 5c51c8593288459a8135b6fc1dc08841 P 7b60df1008ed476f85387868a8aed4d6: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126429568977-17848-0/minicluster-data/ts-2/wal/wals/5c51c8593288459a8135b6fc1dc08841
I20260807 18:14:39.686079 25113 ts_tablet_manager.cc:1939] T 5c51c8593288459a8135b6fc1dc08841 P 7b60df1008ed476f85387868a8aed4d6: Deleting consensus metadata
I20260807 18:14:39.688446 24415 catalog_manager.cc:4841] TS 7b60df1008ed476f85387868a8aed4d6 (127.17.110.3:44669): tablet 5c51c8593288459a8135b6fc1dc08841 (table test-table [id=57b0e6900a084d5a999127200b8c1624]) successfully deleted
I20260807 18:14:39.801340 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 24517
W20260807 18:14:39.837069 24415 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.110.1:37311 (error 108)
I20260807 18:14:39.837958 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 24692
I20260807 18:14:39.839809 25069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:39.839075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:37311 (local address 127.0.0.1:34474)
0807 18:14:39.839440 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:39.839611 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:37311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175}
W20260807 18:14:39.840276 24415 catalog_manager.cc:4584] TS db96c40826094bc4992fee2f1e5896e3 (127.17.110.1:37311): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5c51c8593288459a8135b6fc1dc08841: Network error: Client connection negotiation failed: client connection to 127.17.110.1:37311: connect: Connection refused (error 111)
I20260807 18:14:39.841969 25067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:39.839386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58636 (local address 127.17.110.2:42243)
0807 18:14:39.839592 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:39.839599 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:39.839622 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:14:39.839812 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:39.839816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:39.839876 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:39.840007 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:39.841284 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:39.841290 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:39.841295 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:14:39.841327 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:39.841359 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:14:39.841372 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:39.841530 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:39.841616 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 18:14:39.841683 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 18:14:39.842901 24868 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:14:39.894383 25069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:39.893733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:37311 (local address 127.0.0.1:34486)
0807 18:14:39.894029 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:39.894150 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:37311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260807 18:14:39.895967 25069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:39.895460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:37311 (local address 127.0.0.1:34496)
0807 18:14:39.895655 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:39.895772 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:37311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 18:14:39.957088 25069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:39.956424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:37311 (local address 127.0.0.1:34512)
0807 18:14:39.956752 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:39.956910 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:37311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260807 18:14:39.958441 25069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:39.957999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:37311 (local address 127.0.0.1:34514)
0807 18:14:39.958180 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:39.958281 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:37311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 18:14:39.966856 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 24692
I20260807 18:14:39.996635 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 24882
I20260807 18:14:39.999480 25071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:39.997456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48194 (local address 127.17.110.3:44669)
0807 18:14:39.997651 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:39.997656 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:39.997676 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:14:39.997808 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:39.997812 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:39.997892 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:39.997985 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:39.998952 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:39.998957 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:39.998960 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:14:39.998977 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:39.999011 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:14:39.999023 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:39.999148 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:39.999192 (+ 44us) server_negotiation.cc:299] Negotiation successful
0807 18:14:39.999244 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 18:14:40.000166 25043 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:14:40.039714 25069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:40.039005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:37311 (local address 127.0.0.1:34518)
0807 18:14:40.039377 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:40.039503 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:37311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260807 18:14:40.041047 25069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:40.040576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:37311 (local address 127.0.0.1:34522)
0807 18:14:40.040805 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:40.040893 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:37311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260807 18:14:40.095784 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 24882
I20260807 18:14:40.119683 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 24389
I20260807 18:14:40.120949 24474 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:14:40.233008 25069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:40.232092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:37311 (local address 127.0.0.1:34528)
0807 18:14:40.232576 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:40.232749 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:37311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 18:14:40.233125 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 24389
2026-08-07T18:14:40Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (3044 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 18:14:40.283793 17848 mini_oidc.cc:152] Starting JWKS server
I20260807 18:14:40.286926 17848 webserver.cc:460] Webserver started at https://127.0.0.1:34511/ using document root <none> and password file <none>
I20260807 18:14:40.287317 17848 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 18:14:40.288023 17848 webserver.cc:460] Webserver started at http://127.0.0.1:44587/ using document root <none> and password file <none>
2026-08-07T18:14:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:14:40Z Disabled control of system clock
I20260807 18:14:40.325342 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:34109
--webserver_interface=127.17.110.62
--webserver_port=0
--builtin_ntp_servers=127.17.110.20:33005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:34109
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34511/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126429568977-17848-0/testchainca.cert with env {}
W20260807 18:14:40.680701 25127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:40.681110 25127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:40.681296 25127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:40.689483 25127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:14:40.689702 25127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:40.689788 25127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:40.689899 25127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:14:40.689994 25127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 18:14:40.690096 25127 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 18:14:40.690176 25127 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34511/jwks.json
I20260807 18:14:40.701244 25127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:33005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:34109
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:34109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34511/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126429568977-17848-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:40.702766 25127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:40.704169 25127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:40.716975 25132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:40.716913 25135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:40.716876 25133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:40.718416 25127 server_base.cc:1034] running on GCE node
I20260807 18:14:40.719434 25127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:40.721192 25127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:40.722468 25127 hybrid_clock.cc:648] HybridClock initialized: now 1786126480722435 us; error 58 us; skew 500 ppm
I20260807 18:14:40.722951 25127 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:14:40.725773 25127 webserver.cc:460] Webserver started at http://127.17.110.62:40207/ using document root <none> and password file <none>
I20260807 18:14:40.726624 25127 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:40.726818 25127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:40.727192 25127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:40.730449 25127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "f21691edb51b44d99e33f836bc2562df"
format_stamp: "Formatted at 2026-08-07 18:14:40 on dist-test-slave-39l8"
server_key: "380489f0abc58477e2c2c51344e0c4b1"
server_key_iv: "8dfdc74bfc7c458bfe146c6a9756cb91"
server_key_version: "encryptionkey@0"
I20260807 18:14:40.731500 25127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "f21691edb51b44d99e33f836bc2562df"
format_stamp: "Formatted at 2026-08-07 18:14:40 on dist-test-slave-39l8"
server_key: "380489f0abc58477e2c2c51344e0c4b1"
server_key_iv: "8dfdc74bfc7c458bfe146c6a9756cb91"
server_key_version: "encryptionkey@0"
I20260807 18:14:40.737519 25127 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 18:14:40.741717 25143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:40.743372 25127 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:14:40.743623 25127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "f21691edb51b44d99e33f836bc2562df"
format_stamp: "Formatted at 2026-08-07 18:14:40 on dist-test-slave-39l8"
server_key: "380489f0abc58477e2c2c51344e0c4b1"
server_key_iv: "8dfdc74bfc7c458bfe146c6a9756cb91"
server_key_version: "encryptionkey@0"
I20260807 18:14:40.743888 25127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:40.755617 25127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:40.756623 25127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:40.756991 25127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:40.777335 25127 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:34109
I20260807 18:14:40.777356 25212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:34109 every 8 connection(s)
I20260807 18:14:40.778930 25127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:14:40.782254 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 25127
I20260807 18:14:40.782492 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:14:40.783350 17848 external_mini_cluster.cc:1428] Setting key 122ea3da81efae5dc8e8ef396ecaee9b
I20260807 18:14:40.783690 25213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:40.799237 25213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f21691edb51b44d99e33f836bc2562df: Bootstrap starting.
I20260807 18:14:40.799640 25217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:40.785497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47488 (local address 127.17.110.62:34109)
0807 18:14:40.786596 (+ 1099us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:40.786609 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:40.786658 (+ 49us) server_negotiation.cc:407] Connection header received
0807 18:14:40.788844 (+ 2186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:40.788884 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:40.789505 (+ 621us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:40.789932 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:40.790674 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:40.792243 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:40.794293 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:40.795154 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:40.795336 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:40.796782 (+ 1446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:40.796791 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:40.796809 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:14:40.797434 (+ 625us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:40.797527 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:14:40.797553 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:40.797646 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:40.797971 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 18:14:40.798383 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":397,"threads_started":1}
I20260807 18:14:40.802958 25213 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f21691edb51b44d99e33f836bc2562df: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:40.804103 17848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:14:40.804265 25213 log.cc:824] T 00000000000000000000000000000000 P f21691edb51b44d99e33f836bc2562df: Log is configured to *not* fsync() on all Append() calls
I20260807 18:14:40.807730 25213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f21691edb51b44d99e33f836bc2562df: No bootstrap required, opened a new log
I20260807 18:14:40.815902 25213 raft_consensus.cc:348] T 00000000000000000000000000000000 P f21691edb51b44d99e33f836bc2562df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f21691edb51b44d99e33f836bc2562df" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 34109 } }
I20260807 18:14:40.816397 25213 raft_consensus.cc:374] T 00000000000000000000000000000000 P f21691edb51b44d99e33f836bc2562df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:40.816529 25213 raft_consensus.cc:729] T 00000000000000000000000000000000 P f21691edb51b44d99e33f836bc2562df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f21691edb51b44d99e33f836bc2562df, State: Initialized, Role: FOLLOWER
I20260807 18:14:40.817241 25213 consensus_queue.cc:260] T 00000000000000000000000000000000 P f21691edb51b44d99e33f836bc2562df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f21691edb51b44d99e33f836bc2562df" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 34109 } }
I20260807 18:14:40.817521 25213 raft_consensus.cc:388] T 00000000000000000000000000000000 P f21691edb51b44d99e33f836bc2562df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:14:40.817682 25213 raft_consensus.cc:482] T 00000000000000000000000000000000 P f21691edb51b44d99e33f836bc2562df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:14:40.817883 25213 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f21691edb51b44d99e33f836bc2562df [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:40.820362 25213 raft_consensus.cc:504] T 00000000000000000000000000000000 P f21691edb51b44d99e33f836bc2562df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f21691edb51b44d99e33f836bc2562df" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 34109 } }
I20260807 18:14:40.820968 25213 leader_election.cc:302] T 00000000000000000000000000000000 P f21691edb51b44d99e33f836bc2562df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f21691edb51b44d99e33f836bc2562df; no voters:
I20260807 18:14:40.821728 25213 leader_election.cc:288] T 00000000000000000000000000000000 P f21691edb51b44d99e33f836bc2562df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:14:40.821961 25219 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f21691edb51b44d99e33f836bc2562df [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:14:40.823246 25219 raft_consensus.cc:686] T 00000000000000000000000000000000 P f21691edb51b44d99e33f836bc2562df [term 1 LEADER]: Becoming Leader. State: Replica: f21691edb51b44d99e33f836bc2562df, State: Running, Role: LEADER
I20260807 18:14:40.823868 25219 consensus_queue.cc:237] T 00000000000000000000000000000000 P f21691edb51b44d99e33f836bc2562df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f21691edb51b44d99e33f836bc2562df" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 34109 } }
I20260807 18:14:40.824306 25213 sys_catalog.cc:564] T 00000000000000000000000000000000 P f21691edb51b44d99e33f836bc2562df [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:14:40.827031 25223 sys_catalog.cc:455] T 00000000000000000000000000000000 P f21691edb51b44d99e33f836bc2562df [sys.catalog]: SysCatalogTable state changed. Reason: New leader f21691edb51b44d99e33f836bc2562df. Latest consensus state: current_term: 1 leader_uuid: "f21691edb51b44d99e33f836bc2562df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f21691edb51b44d99e33f836bc2562df" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 34109 } } }
I20260807 18:14:40.827643 25223 sys_catalog.cc:458] T 00000000000000000000000000000000 P f21691edb51b44d99e33f836bc2562df [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:40.827466 25221 sys_catalog.cc:455] T 00000000000000000000000000000000 P f21691edb51b44d99e33f836bc2562df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f21691edb51b44d99e33f836bc2562df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f21691edb51b44d99e33f836bc2562df" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 34109 } } }
I20260807 18:14:40.827804 25221 sys_catalog.cc:458] T 00000000000000000000000000000000 P f21691edb51b44d99e33f836bc2562df [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:40.829110 25228 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:14:40.834651 25228 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:14:40.843856 25228 catalog_manager.cc:1347] Generated new cluster ID: 2989e5a1f6f248aca1640913d6c41691
I20260807 18:14:40.843997 25228 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:14:40.877517 25228 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:14:40.879168 25228 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:14:40.889322 25228 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f21691edb51b44d99e33f836bc2562df: Generated new TSK 0
I20260807 18:14:40.890331 25228 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 18:14:40.944183 25257 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 18:14:40.944451 25257 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 18:14:40.946604 25217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:40.920461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47496 (local address 127.17.110.62:34109)
0807 18:14:40.920763 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:40.920774 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:40.921323 (+ 549us) server_negotiation.cc:407] Connection header received
0807 18:14:40.921783 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:40.921788 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:40.921870 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:40.922020 (+ 150us) server_negotiation.cc:226] Negotiated authn=JWT
0807 18:14:40.922814 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:40.923672 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:40.926005 (+ 2333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:40.926587 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:40.926816 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:40.927585 (+ 769us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 18:14:40.945949 (+ 18364us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34511/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 18:14:40.946337 (+ 388us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47496: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34511/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":91}
W20260807 18:14:40.946789 25217 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47496: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34511/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 18:14:40.949172 25254 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:14:40.920346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:34109 (local address 127.0.0.1:47496)
0807 18:14:40.921155 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:40.921178 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:40.921407 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:40.922080 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:40.922086 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:40.922102 (+ 16us) client_negotiation.cc:189] Negotiated authn=JWT
0807 18:14:40.922562 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:40.922575 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:40.923852 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:40.923859 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:40.925808 (+ 1949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:40.925828 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:40.926860 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:40.926870 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:40.927089 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:40.927187 (+ 98us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 18:14:40.946326 (+ 19139us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34511/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 18:14:40.946792 (+ 466us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.110.62:34109: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34511/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":425,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.17.110.62:34109: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34511/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 18:14:40.951298 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 25127
I20260807 18:14:40.952668 25207 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:14:41.054852 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 25127
2026-08-07T18:14:41Z chronyd exiting
I20260807 18:14:41.109087 17848 test_util.cc:175] -----------------------------------------------
I20260807 18:14:41.109277 17848 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126429568977-17848-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (830 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 18:14:41.113422 17848 mini_oidc.cc:152] Starting JWKS server
I20260807 18:14:41.115487 17848 webserver.cc:460] Webserver started at https://127.0.0.1:34799/ using document root <none> and password file <none>
I20260807 18:14:41.115705 17848 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 18:14:41.116413 17848 webserver.cc:460] Webserver started at http://127.0.0.1:45797/ using document root <none> and password file <none>
2026-08-07T18:14:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:14:41Z Disabled control of system clock
I20260807 18:14:41.153788 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:39871
--webserver_interface=127.17.110.62
--webserver_port=0
--builtin_ntp_servers=127.17.110.20:43571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:39871
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34799/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126429568977-17848-0/testchainca.cert with env {}
W20260807 18:14:41.509289 25265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:41.509616 25265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:41.509786 25265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:41.517539 25265 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:14:41.517722 25265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:41.517794 25265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:41.517889 25265 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:14:41.518164 25265 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 18:14:41.518424 25265 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 18:14:41.518515 25265 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34799/jwks.json
I20260807 18:14:41.529115 25265 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:43571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:39871
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:39871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34799/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126429568977-17848-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:41.530551 25265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:41.531994 25265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:41.544601 25270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:41.544561 25272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:41.544804 25275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:41.545688 25265 server_base.cc:1034] running on GCE node
I20260807 18:14:41.546593 25265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:41.548298 25265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:41.549583 25265 hybrid_clock.cc:648] HybridClock initialized: now 1786126481549536 us; error 98 us; skew 500 ppm
I20260807 18:14:41.550024 25265 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:14:41.552848 25265 webserver.cc:460] Webserver started at http://127.17.110.62:36605/ using document root <none> and password file <none>
I20260807 18:14:41.553776 25265 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:41.553938 25265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:41.554318 25265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:41.557562 25265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "f1f4d061e7dc40449287f980df2e19b6"
format_stamp: "Formatted at 2026-08-07 18:14:41 on dist-test-slave-39l8"
server_key: "4714bb4cbd73aca04abb2e93af39bd3e"
server_key_iv: "308bab4c24a69436be2b77f7895cb697"
server_key_version: "encryptionkey@0"
I20260807 18:14:41.558477 25265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "f1f4d061e7dc40449287f980df2e19b6"
format_stamp: "Formatted at 2026-08-07 18:14:41 on dist-test-slave-39l8"
server_key: "4714bb4cbd73aca04abb2e93af39bd3e"
server_key_iv: "308bab4c24a69436be2b77f7895cb697"
server_key_version: "encryptionkey@0"
I20260807 18:14:41.564503 25265 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 18:14:41.568962 25281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:41.570927 25265 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 18:14:41.571249 25265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "f1f4d061e7dc40449287f980df2e19b6"
format_stamp: "Formatted at 2026-08-07 18:14:41 on dist-test-slave-39l8"
server_key: "4714bb4cbd73aca04abb2e93af39bd3e"
server_key_iv: "308bab4c24a69436be2b77f7895cb697"
server_key_version: "encryptionkey@0"
I20260807 18:14:41.571542 25265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:41.592458 25265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:41.593488 25265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:41.593852 25265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:41.614410 25265 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:39871
I20260807 18:14:41.614424 25356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:39871 every 8 connection(s)
I20260807 18:14:41.615955 25265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:14:41.620292 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 25265
I20260807 18:14:41.620529 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:14:41.620812 25357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:41.621493 17848 external_mini_cluster.cc:1428] Setting key 6d3e91669759868a609104b985139714
I20260807 18:14:41.636554 25357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1f4d061e7dc40449287f980df2e19b6: Bootstrap starting.
I20260807 18:14:41.636734 25361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:41.623882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41380 (local address 127.17.110.62:39871)
0807 18:14:41.625070 (+ 1188us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:41.625082 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:41.625133 (+ 51us) server_negotiation.cc:407] Connection header received
0807 18:14:41.626974 (+ 1841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:41.627003 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:41.627550 (+ 547us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:41.628017 (+ 467us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:41.628679 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:41.630041 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:41.631531 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:41.632319 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:41.632567 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:41.633962 (+ 1395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:41.633972 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:41.633991 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:14:41.634624 (+ 633us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:41.634795 (+ 171us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:14:41.634824 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:41.634976 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:41.635249 (+ 273us) server_negotiation.cc:299] Negotiation successful
0807 18:14:41.635641 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":688,"thread_start_us":421,"threads_started":1}
I20260807 18:14:41.641084 25357 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1f4d061e7dc40449287f980df2e19b6: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:41.641513 17848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:14:41.642416 25357 log.cc:824] T 00000000000000000000000000000000 P f1f4d061e7dc40449287f980df2e19b6: Log is configured to *not* fsync() on all Append() calls
I20260807 18:14:41.645984 25357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1f4d061e7dc40449287f980df2e19b6: No bootstrap required, opened a new log
I20260807 18:14:41.654613 25357 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1f4d061e7dc40449287f980df2e19b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1f4d061e7dc40449287f980df2e19b6" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 39871 } }
I20260807 18:14:41.655123 25357 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1f4d061e7dc40449287f980df2e19b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:41.655265 25357 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1f4d061e7dc40449287f980df2e19b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1f4d061e7dc40449287f980df2e19b6, State: Initialized, Role: FOLLOWER
I20260807 18:14:41.655881 25357 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1f4d061e7dc40449287f980df2e19b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1f4d061e7dc40449287f980df2e19b6" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 39871 } }
I20260807 18:14:41.656113 25357 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1f4d061e7dc40449287f980df2e19b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:14:41.656302 25357 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1f4d061e7dc40449287f980df2e19b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:14:41.656493 25357 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1f4d061e7dc40449287f980df2e19b6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:41.659256 25357 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1f4d061e7dc40449287f980df2e19b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1f4d061e7dc40449287f980df2e19b6" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 39871 } }
I20260807 18:14:41.660054 25357 leader_election.cc:302] T 00000000000000000000000000000000 P f1f4d061e7dc40449287f980df2e19b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1f4d061e7dc40449287f980df2e19b6; no voters:
I20260807 18:14:41.660943 25357 leader_election.cc:288] T 00000000000000000000000000000000 P f1f4d061e7dc40449287f980df2e19b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:14:41.661140 25363 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1f4d061e7dc40449287f980df2e19b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:14:41.662551 25363 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1f4d061e7dc40449287f980df2e19b6 [term 1 LEADER]: Becoming Leader. State: Replica: f1f4d061e7dc40449287f980df2e19b6, State: Running, Role: LEADER
I20260807 18:14:41.663156 25363 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1f4d061e7dc40449287f980df2e19b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1f4d061e7dc40449287f980df2e19b6" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 39871 } }
I20260807 18:14:41.663568 25357 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1f4d061e7dc40449287f980df2e19b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:14:41.666291 25366 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1f4d061e7dc40449287f980df2e19b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1f4d061e7dc40449287f980df2e19b6. Latest consensus state: current_term: 1 leader_uuid: "f1f4d061e7dc40449287f980df2e19b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1f4d061e7dc40449287f980df2e19b6" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 39871 } } }
I20260807 18:14:41.666925 25366 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1f4d061e7dc40449287f980df2e19b6 [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:41.666267 25364 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1f4d061e7dc40449287f980df2e19b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1f4d061e7dc40449287f980df2e19b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1f4d061e7dc40449287f980df2e19b6" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 39871 } } }
I20260807 18:14:41.667687 25364 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1f4d061e7dc40449287f980df2e19b6 [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:41.668356 25378 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:14:41.674474 25378 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:14:41.685946 25378 catalog_manager.cc:1347] Generated new cluster ID: e7a80e0f4ba94708a73102f95fbe2fb1
I20260807 18:14:41.686125 25378 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:14:41.699873 25378 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:14:41.701289 25378 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:14:41.710448 25378 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1f4d061e7dc40449287f980df2e19b6: Generated new TSK 0
I20260807 18:14:41.711699 25378 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 18:14:41.757769 25402 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 18:14:41.757926 25402 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 18:14:41.759265 25361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:41.731047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41386 (local address 127.17.110.62:39871)
0807 18:14:41.731274 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:41.731283 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:41.731706 (+ 423us) server_negotiation.cc:407] Connection header received
0807 18:14:41.731998 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:41.732002 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:41.732095 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:41.732278 (+ 183us) server_negotiation.cc:226] Negotiated authn=JWT
0807 18:14:41.732855 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:41.733631 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:41.735145 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:41.735636 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:41.735811 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:41.736385 (+ 574us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 18:14:41.758364 (+ 21979us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34799/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 18:14:41.758884 (+ 520us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41386: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34799/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":68}
W20260807 18:14:41.759438 25361 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41386: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34799/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260807 18:14:41.759509 25401 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:14:41.730920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:39871 (local address 127.0.0.1:41386)
0807 18:14:41.731568 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:41.731586 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:41.731781 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:41.732287 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:41.732291 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:41.732304 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0807 18:14:41.732659 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:41.732669 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:41.733796 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:41.733809 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:41.734991 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:41.735003 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:41.735847 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:41.735854 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:41.736015 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:41.736074 (+ 59us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 18:14:41.758687 (+ 22613us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34799/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 18:14:41.759027 (+ 340us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.110.62:39871: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34799/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":361,"threads_started":1}
I20260807 18:14:41.761304 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 25265
I20260807 18:14:41.762748 25352 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:14:41.863799 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 25265
2026-08-07T18:14:41Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (818 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 18:14:41.931886 17848 mini_oidc.cc:152] Starting JWKS server
I20260807 18:14:41.933938 17848 webserver.cc:460] Webserver started at https://127.0.0.1:32797/ using document root <none> and password file <none>
I20260807 18:14:41.934181 17848 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 18:14:41.934958 17848 webserver.cc:460] Webserver started at http://127.0.0.1:41423/ using document root <none> and password file <none>
2026-08-07T18:14:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:14:41Z Disabled control of system clock
I20260807 18:14:41.973919 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:43661
--webserver_interface=127.17.110.62
--webserver_port=0
--builtin_ntp_servers=127.17.110.20:42029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:43661
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32797/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 18:14:42.342540 25414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:42.342960 25414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:42.343063 25414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:42.350670 25414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:14:42.350867 25414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:42.350908 25414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:42.350968 25414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:14:42.351014 25414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 18:14:42.351080 25414 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 18:14:42.351114 25414 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:32797/jwks.json
I20260807 18:14:42.361831 25414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:42029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:43661
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:43661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32797/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:42.363349 25414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:42.365276 25414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:42.379137 25421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:42.379230 25422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:42.379230 25424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:42.380635 25414 server_base.cc:1034] running on GCE node
I20260807 18:14:42.381635 25414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:42.383327 25414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:42.384583 25414 hybrid_clock.cc:648] HybridClock initialized: now 1786126482384523 us; error 81 us; skew 500 ppm
I20260807 18:14:42.384979 25414 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:14:42.387789 25414 webserver.cc:460] Webserver started at http://127.17.110.62:42581/ using document root <none> and password file <none>
I20260807 18:14:42.388629 25414 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:42.388789 25414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:42.389178 25414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:42.392889 25414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "3d20fc4b71a84bf9b968563c772bf391"
format_stamp: "Formatted at 2026-08-07 18:14:42 on dist-test-slave-39l8"
server_key: "02911c28d8cbcdb839af84fddb9a564a"
server_key_iv: "3d3d8c057cb65d29e65d7cdb30b5c8b3"
server_key_version: "encryptionkey@0"
I20260807 18:14:42.393831 25414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "3d20fc4b71a84bf9b968563c772bf391"
format_stamp: "Formatted at 2026-08-07 18:14:42 on dist-test-slave-39l8"
server_key: "02911c28d8cbcdb839af84fddb9a564a"
server_key_iv: "3d3d8c057cb65d29e65d7cdb30b5c8b3"
server_key_version: "encryptionkey@0"
I20260807 18:14:42.400110 25414 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 18:14:42.404242 25430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:42.405938 25414 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:14:42.406204 25414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "3d20fc4b71a84bf9b968563c772bf391"
format_stamp: "Formatted at 2026-08-07 18:14:42 on dist-test-slave-39l8"
server_key: "02911c28d8cbcdb839af84fddb9a564a"
server_key_iv: "3d3d8c057cb65d29e65d7cdb30b5c8b3"
server_key_version: "encryptionkey@0"
I20260807 18:14:42.406491 25414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:42.440372 25414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:42.441339 25414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:42.441701 25414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:42.462426 25414 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:43661
I20260807 18:14:42.462541 25506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:43661 every 8 connection(s)
I20260807 18:14:42.463999 25414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:14:42.468334 25507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:42.473167 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 25414
I20260807 18:14:42.473438 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:14:42.474102 17848 external_mini_cluster.cc:1428] Setting key 28bb3602f2e1e7921385aed7f1b07c60
I20260807 18:14:42.481165 25507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d20fc4b71a84bf9b968563c772bf391: Bootstrap starting.
I20260807 18:14:42.485412 25507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d20fc4b71a84bf9b968563c772bf391: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:42.486835 25507 log.cc:824] T 00000000000000000000000000000000 P 3d20fc4b71a84bf9b968563c772bf391: Log is configured to *not* fsync() on all Append() calls
I20260807 18:14:42.490975 25507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d20fc4b71a84bf9b968563c772bf391: No bootstrap required, opened a new log
I20260807 18:14:42.491101 25510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:42.476305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39054 (local address 127.17.110.62:43661)
0807 18:14:42.477589 (+ 1284us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:42.477604 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:42.477656 (+ 52us) server_negotiation.cc:407] Connection header received
0807 18:14:42.479881 (+ 2225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:42.479912 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:42.480494 (+ 582us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:42.480995 (+ 501us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:42.481698 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:42.483301 (+ 1603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:42.484931 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:42.486113 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:42.486434 (+ 321us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:42.488076 (+ 1642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:42.488089 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:42.488113 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:14:42.489054 (+ 941us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:42.489170 (+ 116us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:14:42.489204 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:42.489320 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:42.489578 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 18:14:42.489984 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":750,"thread_start_us":445,"threads_started":1}
I20260807 18:14:42.495930 17848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:14:42.500661 25507 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d20fc4b71a84bf9b968563c772bf391 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d20fc4b71a84bf9b968563c772bf391" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 43661 } }
I20260807 18:14:42.501207 25507 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d20fc4b71a84bf9b968563c772bf391 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:42.501457 25507 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d20fc4b71a84bf9b968563c772bf391 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d20fc4b71a84bf9b968563c772bf391, State: Initialized, Role: FOLLOWER
I20260807 18:14:42.502120 25507 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d20fc4b71a84bf9b968563c772bf391 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d20fc4b71a84bf9b968563c772bf391" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 43661 } }
I20260807 18:14:42.502363 25507 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d20fc4b71a84bf9b968563c772bf391 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:14:42.502487 25507 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d20fc4b71a84bf9b968563c772bf391 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:14:42.502693 25507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d20fc4b71a84bf9b968563c772bf391 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:42.505513 25507 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d20fc4b71a84bf9b968563c772bf391 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d20fc4b71a84bf9b968563c772bf391" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 43661 } }
I20260807 18:14:42.506142 25507 leader_election.cc:302] T 00000000000000000000000000000000 P 3d20fc4b71a84bf9b968563c772bf391 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d20fc4b71a84bf9b968563c772bf391; no voters:
I20260807 18:14:42.506827 25507 leader_election.cc:288] T 00000000000000000000000000000000 P 3d20fc4b71a84bf9b968563c772bf391 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:14:42.507033 25513 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d20fc4b71a84bf9b968563c772bf391 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:14:42.507750 25513 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d20fc4b71a84bf9b968563c772bf391 [term 1 LEADER]: Becoming Leader. State: Replica: 3d20fc4b71a84bf9b968563c772bf391, State: Running, Role: LEADER
I20260807 18:14:42.508571 25513 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d20fc4b71a84bf9b968563c772bf391 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d20fc4b71a84bf9b968563c772bf391" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 43661 } }
I20260807 18:14:42.509924 25507 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d20fc4b71a84bf9b968563c772bf391 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:14:42.511750 25515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d20fc4b71a84bf9b968563c772bf391 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d20fc4b71a84bf9b968563c772bf391" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d20fc4b71a84bf9b968563c772bf391" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 43661 } } }
I20260807 18:14:42.512682 25515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d20fc4b71a84bf9b968563c772bf391 [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:42.512280 25514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d20fc4b71a84bf9b968563c772bf391 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d20fc4b71a84bf9b968563c772bf391. Latest consensus state: current_term: 1 leader_uuid: "3d20fc4b71a84bf9b968563c772bf391" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d20fc4b71a84bf9b968563c772bf391" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 43661 } } }
I20260807 18:14:42.513085 25514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d20fc4b71a84bf9b968563c772bf391 [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:42.515785 25520 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:14:42.521937 25520 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:14:42.532373 25520 catalog_manager.cc:1347] Generated new cluster ID: e731992a625f4a86883669c9a2e0b5ed
I20260807 18:14:42.532608 25520 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:14:42.558676 25520 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:14:42.560807 25520 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:14:42.571693 25520 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d20fc4b71a84bf9b968563c772bf391: Generated new TSK 0
I20260807 18:14:42.573071 25520 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:14:42.603130 25510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:42.593963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39070 (local address 127.17.110.62:43661)
0807 18:14:42.594236 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:42.594246 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:42.594815 (+ 569us) server_negotiation.cc:407] Connection header received
0807 18:14:42.595020 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:42.595025 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:42.595116 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:42.595261 (+ 145us) server_negotiation.cc:226] Negotiated authn=JWT
0807 18:14:42.595858 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:42.596614 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:42.598841 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:42.599350 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:42.599464 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:42.600193 (+ 729us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 18:14:42.602335 (+ 2142us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32797/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 18:14:42.602779 (+ 444us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39070: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32797/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":87}
W20260807 18:14:42.603382 25545 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:14:42.593900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:43661 (local address 127.0.0.1:39070)
0807 18:14:42.594585 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:42.594606 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:42.594814 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:42.595302 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:42.595307 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:42.595321 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0807 18:14:42.595687 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:42.595700 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:42.596780 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:42.596787 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:42.598599 (+ 1812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:42.598618 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:42.599591 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:42.599599 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:42.599807 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:42.599865 (+ 58us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 18:14:42.602584 (+ 2719us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32797/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 18:14:42.602924 (+ 340us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.110.62:43661: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32797/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":376,"threads_started":1}
W20260807 18:14:42.603585 25510 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39070: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32797/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260807 18:14:42.604382 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 25414
I20260807 18:14:42.605890 25500 generic_service.cc:196] Checking for leaks (request via RPC)
E20260807 18:14:42.620721 25546 webserver.cc:533] Webserver: sslize failed:
E20260807 18:14:42.620929 25546 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 18:14:42.707607 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 25414
2026-08-07T18:14:42Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (841 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T18:14:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:14:42Z Disabled control of system clock
I20260807 18:14:42.813777 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:41809
--webserver_interface=127.17.110.62
--webserver_port=0
--builtin_ntp_servers=127.17.110.20:41767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:41809
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126429568977-17848-0/testchainca.cert with env {}
W20260807 18:14:43.176637 25553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:43.176976 25553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:43.177067 25553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:43.184743 25553 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:14:43.184880 25553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:43.184919 25553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:43.184970 25553 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:14:43.185016 25553 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 18:14:43.185089 25553 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 18:14:43.185132 25553 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 18:14:43.195587 25553 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:41767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:41809
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:41809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126429568977-17848-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:43.196956 25553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:43.198309 25553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:43.212623 25561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:43.212710 25563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:43.212669 25560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:43.214144 25553 server_base.cc:1034] running on GCE node
I20260807 18:14:43.215212 25553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:43.216832 25553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:43.218166 25553 hybrid_clock.cc:648] HybridClock initialized: now 1786126483218124 us; error 54 us; skew 500 ppm
I20260807 18:14:43.218621 25553 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:14:43.221414 25553 webserver.cc:460] Webserver started at http://127.17.110.62:33755/ using document root <none> and password file <none>
I20260807 18:14:43.222258 25553 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:43.222419 25553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:43.222867 25553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:43.226094 25553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "8a08618f07e1451fb7e0dc69f391aea5"
format_stamp: "Formatted at 2026-08-07 18:14:43 on dist-test-slave-39l8"
server_key: "1db8569b962c7c9e1fa9ed2a9155e13b"
server_key_iv: "0468cb94e598389ed0938ce4481cd9e4"
server_key_version: "encryptionkey@0"
I20260807 18:14:43.227133 25553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "8a08618f07e1451fb7e0dc69f391aea5"
format_stamp: "Formatted at 2026-08-07 18:14:43 on dist-test-slave-39l8"
server_key: "1db8569b962c7c9e1fa9ed2a9155e13b"
server_key_iv: "0468cb94e598389ed0938ce4481cd9e4"
server_key_version: "encryptionkey@0"
I20260807 18:14:43.233904 25553 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.002s
I20260807 18:14:43.238826 25570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:43.240532 25553 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:14:43.240816 25553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "8a08618f07e1451fb7e0dc69f391aea5"
format_stamp: "Formatted at 2026-08-07 18:14:43 on dist-test-slave-39l8"
server_key: "1db8569b962c7c9e1fa9ed2a9155e13b"
server_key_iv: "0468cb94e598389ed0938ce4481cd9e4"
server_key_version: "encryptionkey@0"
I20260807 18:14:43.241106 25553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:43.257962 25553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:43.259241 25553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:43.259613 25553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:43.280004 25553 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:41809
I20260807 18:14:43.280011 25638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:41809 every 8 connection(s)
I20260807 18:14:43.281735 25553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:14:43.282148 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 25553
I20260807 18:14:43.282361 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:14:43.283361 17848 external_mini_cluster.cc:1428] Setting key 37927cb1bc0656b43583c700bb7fcb11
I20260807 18:14:43.287035 25640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:43.300621 25643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:43.285530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51876 (local address 127.17.110.62:41809)
0807 18:14:43.286622 (+ 1092us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:43.286638 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:43.286722 (+ 84us) server_negotiation.cc:407] Connection header received
0807 18:14:43.289047 (+ 2325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:43.289079 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:43.289717 (+ 638us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:43.290202 (+ 485us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:43.290938 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:43.293082 (+ 2144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:43.294589 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:43.295742 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:43.295951 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:43.297273 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:43.297288 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:43.297313 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:14:43.298154 (+ 841us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:43.298263 (+ 109us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:14:43.298293 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:43.298429 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:43.298729 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 18:14:43.299148 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":324,"threads_started":1}
I20260807 18:14:43.302505 25640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a08618f07e1451fb7e0dc69f391aea5: Bootstrap starting.
I20260807 18:14:43.305330 17848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:14:43.306516 25640 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a08618f07e1451fb7e0dc69f391aea5: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:43.307711 25640 log.cc:824] T 00000000000000000000000000000000 P 8a08618f07e1451fb7e0dc69f391aea5: Log is configured to *not* fsync() on all Append() calls
I20260807 18:14:43.311084 25640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a08618f07e1451fb7e0dc69f391aea5: No bootstrap required, opened a new log
I20260807 18:14:43.320123 25640 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a08618f07e1451fb7e0dc69f391aea5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a08618f07e1451fb7e0dc69f391aea5" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 41809 } }
I20260807 18:14:43.320612 25640 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a08618f07e1451fb7e0dc69f391aea5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:43.320739 25640 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a08618f07e1451fb7e0dc69f391aea5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a08618f07e1451fb7e0dc69f391aea5, State: Initialized, Role: FOLLOWER
I20260807 18:14:43.321455 25640 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a08618f07e1451fb7e0dc69f391aea5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a08618f07e1451fb7e0dc69f391aea5" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 41809 } }
I20260807 18:14:43.321736 25640 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a08618f07e1451fb7e0dc69f391aea5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:14:43.321904 25640 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a08618f07e1451fb7e0dc69f391aea5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:14:43.322121 25640 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a08618f07e1451fb7e0dc69f391aea5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:43.324920 25640 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a08618f07e1451fb7e0dc69f391aea5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a08618f07e1451fb7e0dc69f391aea5" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 41809 } }
I20260807 18:14:43.325639 25640 leader_election.cc:302] T 00000000000000000000000000000000 P 8a08618f07e1451fb7e0dc69f391aea5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a08618f07e1451fb7e0dc69f391aea5; no voters:
I20260807 18:14:43.326227 25640 leader_election.cc:288] T 00000000000000000000000000000000 P 8a08618f07e1451fb7e0dc69f391aea5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:14:43.326414 25647 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a08618f07e1451fb7e0dc69f391aea5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:14:43.327268 25647 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a08618f07e1451fb7e0dc69f391aea5 [term 1 LEADER]: Becoming Leader. State: Replica: 8a08618f07e1451fb7e0dc69f391aea5, State: Running, Role: LEADER
I20260807 18:14:43.327904 25647 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a08618f07e1451fb7e0dc69f391aea5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a08618f07e1451fb7e0dc69f391aea5" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 41809 } }
I20260807 18:14:43.328675 25640 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a08618f07e1451fb7e0dc69f391aea5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:14:43.331135 25648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a08618f07e1451fb7e0dc69f391aea5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a08618f07e1451fb7e0dc69f391aea5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a08618f07e1451fb7e0dc69f391aea5" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 41809 } } }
I20260807 18:14:43.331622 25648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a08618f07e1451fb7e0dc69f391aea5 [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:43.331162 25649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a08618f07e1451fb7e0dc69f391aea5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a08618f07e1451fb7e0dc69f391aea5. Latest consensus state: current_term: 1 leader_uuid: "8a08618f07e1451fb7e0dc69f391aea5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a08618f07e1451fb7e0dc69f391aea5" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 41809 } } }
I20260807 18:14:43.333228 25649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a08618f07e1451fb7e0dc69f391aea5 [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:43.333441 25656 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:14:43.338304 25656 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:14:43.349920 25656 catalog_manager.cc:1347] Generated new cluster ID: 46a7f5a694bb44e19401a6b56553cd4b
I20260807 18:14:43.350067 25656 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:14:43.360883 25656 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:14:43.362624 25656 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:14:43.371258 25656 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a08618f07e1451fb7e0dc69f391aea5: Generated new TSK 0
I20260807 18:14:43.372287 25656 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:14:43.505959 25643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:43.386012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51892 (local address 127.17.110.62:41809)
0807 18:14:43.386281 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:43.386291 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:43.386782 (+ 491us) server_negotiation.cc:407] Connection header received
0807 18:14:43.387123 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:43.387129 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:43.387211 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:43.387365 (+ 154us) server_negotiation.cc:226] Negotiated authn=JWT
0807 18:14:43.387983 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:43.388714 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:43.390893 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:43.391476 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:43.391601 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:43.392283 (+ 682us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 18:14:43.505277 (+112994us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 18:14:43.505687 (+ 410us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51892: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":84}
W20260807 18:14:43.506165 25643 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51892: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260807 18:14:43.506880 25684 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:14:43.385931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:41809 (local address 127.0.0.1:51892)
0807 18:14:43.386625 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:43.386649 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:43.386865 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:43.387421 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:43.387427 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:43.387441 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0807 18:14:43.387779 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:43.387795 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:43.388870 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:43.388876 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:43.390683 (+ 1807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:43.390737 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:43.391698 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:43.391707 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:43.391901 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:43.391958 (+ 57us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 18:14:43.505667 (+113709us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 18:14:43.506074 (+ 407us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.110.62:41809: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":376,"threads_started":1}
I20260807 18:14:43.509145 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 25553
I20260807 18:14:43.510545 25634 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:14:43.616451 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 25553
2026-08-07T18:14:43Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (889 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (462 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (1738 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:14:45 dist-test-slave-39l8 krb5kdc[25704](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:14:45 dist-test-slave-39l8 krb5kdc[25704](info): set up 2 sockets
Aug 07 18:14:45 dist-test-slave-39l8 krb5kdc[25704](info): commencing operation
krb5kdc: starting...
W20260807 18:14:47.187506 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.322s user 0.002s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:14:47 dist-test-slave-39l8 krb5kdc[25704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126487, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:14:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:14:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.110.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
I20260807 18:14:47.462384 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:45465
--webserver_interface=127.17.110.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:44189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:45465
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:14:47.831607 25722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:47.831929 25722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:47.832043 25722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:47.839651 25722 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:14:47.839776 25722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:47.839823 25722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:47.839881 25722 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:14:47.839929 25722 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:14:47.850394 25722 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:44189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:45465
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:45465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:47.851890 25722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:47.853322 25722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:47.866600 25733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:47.866636 25729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:47.866722 25730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:47.867962 25722 server_base.cc:1034] running on GCE node
I20260807 18:14:47.868819 25722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:47.870327 25722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:47.871553 25722 hybrid_clock.cc:648] HybridClock initialized: now 1786126487871506 us; error 44 us; skew 500 ppm
I20260807 18:14:47.871915 25722 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:47 dist-test-slave-39l8 krb5kdc[25704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126487, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:47.875469 25722 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:14:47.877221 25722 webserver.cc:460] Webserver started at http://127.17.110.62:37497/ using document root <none> and password file <none>
I20260807 18:14:47.877974 25722 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:47.878093 25722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:47.878383 25722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:47.881723 25722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "33427dce44ef4d04a4f06d1afae77a73"
format_stamp: "Formatted at 2026-08-07 18:14:47 on dist-test-slave-39l8"
server_key: "cae5c6fa3fe9c51984937c12f00e9ebd"
server_key_iv: "28adca90edbe1417547ebacfeda4fb2b"
server_key_version: "encryptionkey@0"
I20260807 18:14:47.882535 25722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "33427dce44ef4d04a4f06d1afae77a73"
format_stamp: "Formatted at 2026-08-07 18:14:47 on dist-test-slave-39l8"
server_key: "cae5c6fa3fe9c51984937c12f00e9ebd"
server_key_iv: "28adca90edbe1417547ebacfeda4fb2b"
server_key_version: "encryptionkey@0"
I20260807 18:14:47.888722 25722 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260807 18:14:47.892683 25744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:47.894338 25722 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:14:47.894520 25722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "33427dce44ef4d04a4f06d1afae77a73"
format_stamp: "Formatted at 2026-08-07 18:14:47 on dist-test-slave-39l8"
server_key: "cae5c6fa3fe9c51984937c12f00e9ebd"
server_key_iv: "28adca90edbe1417547ebacfeda4fb2b"
server_key_version: "encryptionkey@0"
I20260807 18:14:47.894790 25722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:47.916684 25722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:47.921857 25722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:47.922133 25722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:47.942689 25722 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:45465
I20260807 18:14:47.942826 25816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:45465 every 8 connection(s)
I20260807 18:14:47.944298 25722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:14:47.948633 25817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:47.951907 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 25722
I20260807 18:14:47.952163 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:14:47.952919 17848 external_mini_cluster.cc:1428] Setting key e0cfecd015c3ef33aeb95638da24b497
I20260807 18:14:47.963673 25817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33427dce44ef4d04a4f06d1afae77a73: Bootstrap starting.
I20260807 18:14:47.967798 25817 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33427dce44ef4d04a4f06d1afae77a73: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:47.969023 25817 log.cc:824] T 00000000000000000000000000000000 P 33427dce44ef4d04a4f06d1afae77a73: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:14:47 dist-test-slave-39l8 krb5kdc[25704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126487, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:47.973536 25817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33427dce44ef4d04a4f06d1afae77a73: No bootstrap required, opened a new log
I20260807 18:14:47.980791 25820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:47.955500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32924 (local address 127.17.110.62:45465)
0807 18:14:47.956449 (+ 949us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:47.956467 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:47.956523 (+ 56us) server_negotiation.cc:407] Connection header received
0807 18:14:47.958869 (+ 2346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:47.958918 (+ 49us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:47.959599 (+ 681us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:47.960154 (+ 555us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:47.961731 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:47.963887 (+ 2156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:47.965469 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:47.966598 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:47.966870 (+ 272us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:47.972151 (+ 5281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:47.972164 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:47.972183 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:47.972232 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:47.975810 (+ 3578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:47.976858 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:47.976866 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:47.976874 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:47.977072 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:47.977544 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:47.977548 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:47.977550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:47.978177 (+ 627us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:47.978422 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:47.979067 (+ 645us) server_negotiation.cc:299] Negotiation successful
0807 18:14:47.979468 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":406,"threads_started":1}
I20260807 18:14:47.983870 25817 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33427dce44ef4d04a4f06d1afae77a73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33427dce44ef4d04a4f06d1afae77a73" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45465 } }
I20260807 18:14:47.984373 25817 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33427dce44ef4d04a4f06d1afae77a73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:47.984510 25817 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33427dce44ef4d04a4f06d1afae77a73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33427dce44ef4d04a4f06d1afae77a73, State: Initialized, Role: FOLLOWER
I20260807 18:14:47.985124 25817 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33427dce44ef4d04a4f06d1afae77a73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33427dce44ef4d04a4f06d1afae77a73" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45465 } }
I20260807 18:14:47.985409 25817 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33427dce44ef4d04a4f06d1afae77a73 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:14:47.985569 25817 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33427dce44ef4d04a4f06d1afae77a73 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:14:47.985756 25817 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33427dce44ef4d04a4f06d1afae77a73 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:47.988600 25817 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33427dce44ef4d04a4f06d1afae77a73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33427dce44ef4d04a4f06d1afae77a73" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45465 } }
I20260807 18:14:47.989219 25817 leader_election.cc:302] T 00000000000000000000000000000000 P 33427dce44ef4d04a4f06d1afae77a73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33427dce44ef4d04a4f06d1afae77a73; no voters:
I20260807 18:14:47.989862 25817 leader_election.cc:288] T 00000000000000000000000000000000 P 33427dce44ef4d04a4f06d1afae77a73 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:14:47.990029 25827 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33427dce44ef4d04a4f06d1afae77a73 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:14:47.990800 25827 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33427dce44ef4d04a4f06d1afae77a73 [term 1 LEADER]: Becoming Leader. State: Replica: 33427dce44ef4d04a4f06d1afae77a73, State: Running, Role: LEADER
I20260807 18:14:47.991395 25827 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33427dce44ef4d04a4f06d1afae77a73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33427dce44ef4d04a4f06d1afae77a73" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45465 } }
I20260807 18:14:47.992235 25817 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33427dce44ef4d04a4f06d1afae77a73 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:14:47.995352 25829 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33427dce44ef4d04a4f06d1afae77a73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33427dce44ef4d04a4f06d1afae77a73. Latest consensus state: current_term: 1 leader_uuid: "33427dce44ef4d04a4f06d1afae77a73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33427dce44ef4d04a4f06d1afae77a73" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45465 } } }
I20260807 18:14:47.995798 25829 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33427dce44ef4d04a4f06d1afae77a73 [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:47.996351 25828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33427dce44ef4d04a4f06d1afae77a73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33427dce44ef4d04a4f06d1afae77a73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33427dce44ef4d04a4f06d1afae77a73" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45465 } } }
I20260807 18:14:47.996695 25828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33427dce44ef4d04a4f06d1afae77a73 [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:47.998385 25835 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:14:48.005342 25835 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:14:48.019659 25835 catalog_manager.cc:1347] Generated new cluster ID: 39dffb047e5e4fe0922dada9dc03d9f4
I20260807 18:14:48.019817 25835 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:14:48.030846 25835 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:14:48.032902 25835 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:14:48.044581 25835 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33427dce44ef4d04a4f06d1afae77a73: Generated new TSK 0
I20260807 18:14:48.045818 25835 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.110.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
I20260807 18:14:48.107880 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.1:0
--local_ip_for_outbound_sockets=127.17.110.1
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:45465
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:44189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:14:48.476524 25863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:48.476850 25863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:48.477056 25863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:48.484928 25863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:48.485073 25863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:48.485262 25863 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.1
I20260807 18:14:48.496583 25863 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:44189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:45465
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.1
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:48.498131 25863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:48.500064 25863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:48.515307 25872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:48.515260 25873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:48.515219 25876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:48.516479 25863 server_base.cc:1034] running on GCE node
I20260807 18:14:48.517185 25863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:48.518312 25863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:48.519546 25863 hybrid_clock.cc:648] HybridClock initialized: now 1786126488519485 us; error 49 us; skew 500 ppm
I20260807 18:14:48.519876 25863 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:48 dist-test-slave-39l8 krb5kdc[25704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126488, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:48.523521 25863 init.cc:377] Logged in from keytab as kudu/127.17.110.1@KRBTEST.COM (short username kudu)
I20260807 18:14:48.525202 25863 webserver.cc:460] Webserver started at http://127.17.110.1:43117/ using document root <none> and password file <none>
I20260807 18:14:48.525930 25863 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:48.526042 25863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:48.526332 25863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:48.529692 25863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-0/data/instance:
uuid: "67ce8aa153d7425cb965799621fb70b5"
format_stamp: "Formatted at 2026-08-07 18:14:48 on dist-test-slave-39l8"
server_key: "18ad3837b4b77c17f7a2962a1acad68e"
server_key_iv: "9288452ffea76122a776301b196393b1"
server_key_version: "encryptionkey@0"
I20260807 18:14:48.530568 25863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance:
uuid: "67ce8aa153d7425cb965799621fb70b5"
format_stamp: "Formatted at 2026-08-07 18:14:48 on dist-test-slave-39l8"
server_key: "18ad3837b4b77c17f7a2962a1acad68e"
server_key_iv: "9288452ffea76122a776301b196393b1"
server_key_version: "encryptionkey@0"
I20260807 18:14:48.536827 25863 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260807 18:14:48.540932 25890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:48.542613 25863 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:14:48.542936 25863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-0/wal
uuid: "67ce8aa153d7425cb965799621fb70b5"
format_stamp: "Formatted at 2026-08-07 18:14:48 on dist-test-slave-39l8"
server_key: "18ad3837b4b77c17f7a2962a1acad68e"
server_key_iv: "9288452ffea76122a776301b196393b1"
server_key_version: "encryptionkey@0"
I20260807 18:14:48.543155 25863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:48.559070 25863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:48.563970 25863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:48.564327 25863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:48.565395 25863 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:14:48.566968 25863 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:14:48.567057 25863 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:48.567153 25863 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:14:48.567239 25863 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:48.604427 25863 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.1:45417
I20260807 18:14:48.604636 26042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.1:45417 every 8 connection(s)
I20260807 18:14:48.605834 25863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
I20260807 18:14:48.608690 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 25863
I20260807 18:14:48.608923 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance
I20260807 18:14:48.609833 17848 external_mini_cluster.cc:1428] Setting key 3287121d9e9d563ddd88bc0030e0fca4
Aug 07 18:14:48 dist-test-slave-39l8 krb5kdc[25704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126488, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:48.634053 26045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:48.608858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:36901 (local address 127.17.110.62:45465)
0807 18:14:48.612192 (+ 3334us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:48.612200 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:48.612225 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:14:48.614225 (+ 2000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:48.614230 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:48.614320 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:48.614419 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:48.616950 (+ 2531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:48.617753 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:48.620579 (+ 2826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:48.621016 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:48.621139 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:48.625571 (+ 4432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:48.625583 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:48.625592 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:48.625648 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:48.629696 (+ 4048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:48.630775 (+ 1079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:48.630785 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:48.630791 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:48.630972 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:48.631631 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:48.631640 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:48.631647 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:48.632106 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:48.632297 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:48.633184 (+ 887us) server_negotiation.cc:299] Negotiation successful
0807 18:14:48.633525 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3199,"thread_start_us":3055,"threads_started":1}
I20260807 18:14:48.634869 26046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:48.609514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45465 (local address 127.17.110.1:36901)
0807 18:14:48.611343 (+ 1829us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:48.611413 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:48.613846 (+ 2433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:48.614790 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:48.614807 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:48.615715 (+ 908us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:48.616672 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:48.616706 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:48.617975 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:48.617984 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:48.620364 (+ 2380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:48.620388 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:48.621299 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:48.621310 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:48.621675 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:48.622573 (+ 898us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:48.622610 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:48.625020 (+ 2410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:48.629938 (+ 4918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:48.629949 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:48.629966 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:48.630555 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:48.631208 (+ 653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:48.631219 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:48.631222 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:48.631471 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:48.632283 (+ 812us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:48.632291 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:48.632703 (+ 412us) client_negotiation.cc:769] Sending connection context
0807 18:14:48.632977 (+ 274us) client_negotiation.cc:240] Negotiation successful
0807 18:14:48.633471 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1506,"thread_start_us":1225,"threads_started":1}
I20260807 18:14:48.636734 26043 heartbeater.cc:347] Connected to a master server at 127.17.110.62:45465
I20260807 18:14:48.637095 26043 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:48.637956 26043 heartbeater.cc:511] Master 127.17.110.62:45465 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.17.110.2@KRBTEST.COM; defaulting to no policy
I20260807 18:14:48.640555 25770 ts_manager.cc:194] Registered new tserver with Master: 67ce8aa153d7425cb965799621fb70b5 (127.17.110.1:45417)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.2@KRBTEST.COM" created.
I20260807 18:14:48.642422 25770 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.1@KRBTEST.COM'} at 127.17.110.1:36901
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
I20260807 18:14:48.672788 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.2:0
--local_ip_for_outbound_sockets=127.17.110.2
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:45465
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:44189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:14:49.045274 26049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:49.045593 26049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:49.045760 26049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:49.053637 26049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:49.053782 26049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:49.053968 26049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.2
I20260807 18:14:49.064530 26049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:44189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:45465
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.2
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:49.066015 26049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:49.067515 26049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:49.081863 26057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:49.081920 26059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:49.081935 26055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:49.082806 26049 server_base.cc:1034] running on GCE node
I20260807 18:14:49.083624 26049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:49.084730 26049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:49.085997 26049 hybrid_clock.cc:648] HybridClock initialized: now 1786126489085919 us; error 67 us; skew 500 ppm
I20260807 18:14:49.086402 26049 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:49 dist-test-slave-39l8 krb5kdc[25704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126489, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:49.090056 26049 init.cc:377] Logged in from keytab as kudu/127.17.110.2@KRBTEST.COM (short username kudu)
I20260807 18:14:49.091903 26049 webserver.cc:460] Webserver started at http://127.17.110.2:45647/ using document root <none> and password file <none>
I20260807 18:14:49.092804 26049 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:49.092974 26049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:49.093338 26049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:49.096688 26049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-1/data/instance:
uuid: "a8a4316df058440b8a902e4de60272a9"
format_stamp: "Formatted at 2026-08-07 18:14:49 on dist-test-slave-39l8"
server_key: "2baf39b2bc3e9f36277e930ce6b40557"
server_key_iv: "f6681cdf224113f0a98c1de5dff05877"
server_key_version: "encryptionkey@0"
I20260807 18:14:49.097602 26049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance:
uuid: "a8a4316df058440b8a902e4de60272a9"
format_stamp: "Formatted at 2026-08-07 18:14:49 on dist-test-slave-39l8"
server_key: "2baf39b2bc3e9f36277e930ce6b40557"
server_key_iv: "f6681cdf224113f0a98c1de5dff05877"
server_key_version: "encryptionkey@0"
I20260807 18:14:49.104019 26049 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260807 18:14:49.108155 26067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:49.109845 26049 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:14:49.110074 26049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-1/wal
uuid: "a8a4316df058440b8a902e4de60272a9"
format_stamp: "Formatted at 2026-08-07 18:14:49 on dist-test-slave-39l8"
server_key: "2baf39b2bc3e9f36277e930ce6b40557"
server_key_iv: "f6681cdf224113f0a98c1de5dff05877"
server_key_version: "encryptionkey@0"
I20260807 18:14:49.110337 26049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:49.126035 26049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:49.131340 26049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:49.131749 26049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:49.133016 26049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:14:49.134846 26049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:14:49.134971 26049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:49.135149 26049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:14:49.135244 26049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:49.175529 26049 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.2:43317
I20260807 18:14:49.175575 26233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.2:43317 every 8 connection(s)
I20260807 18:14:49.177119 26049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
I20260807 18:14:49.180903 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 26049
I20260807 18:14:49.181146 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance
I20260807 18:14:49.181804 17848 external_mini_cluster.cc:1428] Setting key 018513989614b51c0d54b926cc9e2f7d
Aug 07 18:14:49 dist-test-slave-39l8 krb5kdc[25704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126489, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:49.201498 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:49.180027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:35873 (local address 127.17.110.62:45465)
0807 18:14:49.180569 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:49.180575 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:49.181841 (+ 1266us) server_negotiation.cc:407] Connection header received
0807 18:14:49.184055 (+ 2214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:49.184062 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:49.184164 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:49.184297 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:49.186077 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:49.187003 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:49.188848 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:49.189346 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:49.189517 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:49.194060 (+ 4543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:49.194067 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:49.194071 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:49.194114 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:49.197235 (+ 3121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:49.198178 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:49.198186 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:49.198190 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:49.198372 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:49.199079 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:49.199084 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:49.199086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:49.199533 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:49.199729 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:49.200769 (+ 1040us) server_negotiation.cc:299] Negotiation successful
0807 18:14:49.201072 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":303,"threads_started":1}
I20260807 18:14:49.202688 26239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:49.180600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45465 (local address 127.17.110.2:35873)
0807 18:14:49.181630 (+ 1030us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:49.181676 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:49.183728 (+ 2052us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:49.184586 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:49.184596 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:49.185178 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:49.185858 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:49.185878 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:49.187213 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:49.187217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:49.188655 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:49.188671 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:49.189641 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:49.189649 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:49.189938 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:49.190952 (+ 1014us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:49.190982 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:49.193742 (+ 2760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:49.197446 (+ 3704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:49.197458 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:49.197480 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:49.198020 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:49.198588 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:49.198592 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:49.198595 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:49.198916 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:49.199707 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:49.199717 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:49.200288 (+ 571us) client_negotiation.cc:769] Sending connection context
0807 18:14:49.200625 (+ 337us) client_negotiation.cc:240] Negotiation successful
0807 18:14:49.201297 (+ 672us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":742,"thread_start_us":472,"threads_started":1}
I20260807 18:14:49.204577 26235 heartbeater.cc:347] Connected to a master server at 127.17.110.62:45465
I20260807 18:14:49.205016 26235 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:49.206104 26235 heartbeater.cc:511] Master 127.17.110.62:45465 requested a full tablet report, sending...
I20260807 18:14:49.208248 25770 ts_manager.cc:194] Registered new tserver with Master: a8a4316df058440b8a902e4de60272a9 (127.17.110.2:43317)
WARNING: no policy specified for kudu/127.17.110.3@KRBTEST.COM; defaulting to no policy
I20260807 18:14:49.209589 25770 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.2@KRBTEST.COM'} at 127.17.110.2:35873
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
I20260807 18:14:49.246361 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.3:0
--local_ip_for_outbound_sockets=127.17.110.3
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:45465
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:44189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:14:49.608289 26244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:49.608566 26244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:49.608727 26244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:49.616825 26244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:49.616971 26244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:49.617156 26244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.3
I20260807 18:14:49.627831 26244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:44189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:45465
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.3
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:49.629278 26244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:49.630836 26244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:49.645460 26250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:49.645520 26251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:49.645478 26254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:49.646288 26043 heartbeater.cc:503] Master 127.17.110.62:45465 was elected leader, sending a full tablet report...
I20260807 18:14:49.646970 26244 server_base.cc:1034] running on GCE node
I20260807 18:14:49.647740 26244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:49.649120 26244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:49.650585 26244 hybrid_clock.cc:648] HybridClock initialized: now 1786126489650480 us; error 66 us; skew 500 ppm
I20260807 18:14:49.650992 26244 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:49 dist-test-slave-39l8 krb5kdc[25704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126489, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:49.654961 26244 init.cc:377] Logged in from keytab as kudu/127.17.110.3@KRBTEST.COM (short username kudu)
I20260807 18:14:49.656854 26244 webserver.cc:460] Webserver started at http://127.17.110.3:42297/ using document root <none> and password file <none>
I20260807 18:14:49.657717 26244 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:49.657902 26244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:49.658288 26244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:49.661902 26244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-2/data/instance:
uuid: "2404a67f2919445eb5cb93b038448c1b"
format_stamp: "Formatted at 2026-08-07 18:14:49 on dist-test-slave-39l8"
server_key: "c25890b759434bf116555bacb6dafd4d"
server_key_iv: "54c390edc466b78ebe9cef86f948af3d"
server_key_version: "encryptionkey@0"
I20260807 18:14:49.662968 26244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance:
uuid: "2404a67f2919445eb5cb93b038448c1b"
format_stamp: "Formatted at 2026-08-07 18:14:49 on dist-test-slave-39l8"
server_key: "c25890b759434bf116555bacb6dafd4d"
server_key_iv: "54c390edc466b78ebe9cef86f948af3d"
server_key_version: "encryptionkey@0"
I20260807 18:14:49.669693 26244 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 18:14:49.674459 26270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:49.676342 26244 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260807 18:14:49.676618 26244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-2/wal
uuid: "2404a67f2919445eb5cb93b038448c1b"
format_stamp: "Formatted at 2026-08-07 18:14:49 on dist-test-slave-39l8"
server_key: "c25890b759434bf116555bacb6dafd4d"
server_key_iv: "54c390edc466b78ebe9cef86f948af3d"
server_key_version: "encryptionkey@0"
I20260807 18:14:49.676911 26244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:49.696509 26244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:49.703001 26244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:49.703372 26244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:49.704483 26244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:14:49.706259 26244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:14:49.706383 26244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:49.706559 26244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:14:49.706650 26244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:49.743543 26244 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.3:36861
I20260807 18:14:49.743597 26429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.3:36861 every 8 connection(s)
I20260807 18:14:49.745169 26244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
I20260807 18:14:49.747265 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 26244
I20260807 18:14:49.747476 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance
I20260807 18:14:49.748064 17848 external_mini_cluster.cc:1428] Setting key e872ba9d736961db3c7f71869cf0d767
Aug 07 18:14:49 dist-test-slave-39l8 krb5kdc[25704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126489, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:14:49.769016 26433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:49.748810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:48197 (local address 127.17.110.62:45465)
0807 18:14:49.749615 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:49.749621 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:49.750817 (+ 1196us) server_negotiation.cc:407] Connection header received
0807 18:14:49.752997 (+ 2180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:49.753001 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:49.753075 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:49.753175 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:49.755118 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:49.755962 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:49.757724 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:49.758149 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:49.758258 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:49.762192 (+ 3934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:49.762198 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:49.762202 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:49.762243 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:49.765513 (+ 3270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:49.766481 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:49.766485 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:49.766487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:49.766625 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:49.767178 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:49.767182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:49.767184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:49.767476 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:49.767622 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:49.768530 (+ 908us) server_negotiation.cc:299] Negotiation successful
0807 18:14:49.768725 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":683,"thread_start_us":534,"threads_started":1}
I20260807 18:14:49.770195 26436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:49.749534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45465 (local address 127.17.110.3:48197)
0807 18:14:49.750610 (+ 1076us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:49.750686 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:49.752694 (+ 2008us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:49.753436 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:49.753446 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:49.754158 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:49.754932 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:49.754955 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:49.756161 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:49.756166 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:49.757562 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:49.757577 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:49.758391 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:49.758396 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:49.758614 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:49.759407 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:49.759432 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:49.761895 (+ 2463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:49.765721 (+ 3826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:49.765732 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:49.765749 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:49.766346 (+ 597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:49.766791 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:49.766795 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:49.766798 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:49.767010 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:49.767647 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:49.767656 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:49.768065 (+ 409us) client_negotiation.cc:769] Sending connection context
0807 18:14:49.768399 (+ 334us) client_negotiation.cc:240] Negotiation successful
0807 18:14:49.768915 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"thread_start_us":458,"threads_started":1}
I20260807 18:14:49.772181 26431 heartbeater.cc:347] Connected to a master server at 127.17.110.62:45465
I20260807 18:14:49.772543 26431 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:49.773360 26431 heartbeater.cc:511] Master 127.17.110.62:45465 requested a full tablet report, sending...
I20260807 18:14:49.775194 25770 ts_manager.cc:194] Registered new tserver with Master: 2404a67f2919445eb5cb93b038448c1b (127.17.110.3:36861)
I20260807 18:14:49.776477 25770 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.3@KRBTEST.COM'} at 127.17.110.3:48197
I20260807 18:14:49.777784 17848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/test-admin.keytab.
Aug 07 18:14:49 dist-test-slave-39l8 krb5kdc[25704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:49.812817 17848 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 18:14:49.818990 26446 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:14:49.816130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46196 (local address 127.17.110.1:45417)
0807 18:14:49.816931 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:49.816938 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:49.816965 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:14:49.817026 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:49.817033 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:49.817270 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:49.817455 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:49.818482 (+ 1027us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46196: BlockingRecv error: recv got EOF from 127.0.0.1:46196 (error 108)
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":410,"threads_started":1}
W20260807 18:14:49.819182 17848 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/krb5cc' of type FILE
Aug 07 18:14:49 dist-test-slave-39l8 krb5kdc[25704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:49.820996 17848 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 18:14:49 dist-test-slave-39l8 krb5kdc[25704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:49.822597 17848 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 18:14:49 dist-test-slave-39l8 krb5kdc[25704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.1@KRBTEST.COM
I20260807 18:14:49.840009 26446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:49.824613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46212 (local address 127.17.110.1:45417)
0807 18:14:49.824876 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:49.824883 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:49.825031 (+ 148us) server_negotiation.cc:407] Connection header received
0807 18:14:49.825202 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:49.825206 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:49.825291 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:49.825388 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:49.826518 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:49.827614 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:49.828999 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:49.829634 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:49.829782 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:49.833890 (+ 4108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:49.833907 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:49.833926 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:49.833969 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:49.836997 (+ 3028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:49.837821 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:49.837828 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:49.837831 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:49.837965 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:49.838336 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:49.838340 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:49.838342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:49.838855 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:49.839098 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:49.839510 (+ 412us) server_negotiation.cc:299] Negotiation successful
0807 18:14:49.839737 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260807 18:14:49.846894 26446 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:14:49.844673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46224 (local address 127.17.110.1:45417)
0807 18:14:49.844882 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:49.844886 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:49.845251 (+ 365us) server_negotiation.cc:407] Connection header received
0807 18:14:49.845407 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:49.845410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:49.845489 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:49.845580 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:49.846512 (+ 932us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46224: BlockingRecv error: recv got EOF from 127.0.0.1:46224 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260807 18:14:49.847191 17848 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/krb5cc' of type FILE
Aug 07 18:14:49 dist-test-slave-39l8 krb5kdc[25704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:49.849027 17848 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 18:14:49 dist-test-slave-39l8 krb5kdc[25704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:49.850594 17848 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 18:14:49 dist-test-slave-39l8 krb5kdc[25704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.1@KRBTEST.COM
I20260807 18:14:49.866648 26446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:49.852247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46228 (local address 127.17.110.1:45417)
0807 18:14:49.852491 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:49.852499 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:49.852878 (+ 379us) server_negotiation.cc:407] Connection header received
0807 18:14:49.853100 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:49.853105 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:49.853172 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:49.853284 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:49.854356 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:49.855152 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:49.856440 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:49.856912 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:49.857083 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:49.861112 (+ 4029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:49.861118 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:49.861122 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:49.861175 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:49.863796 (+ 2621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:49.864473 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:49.864477 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:49.864479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:49.864613 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:49.865095 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:49.865098 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:49.865100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:49.865393 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:49.865604 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:49.865965 (+ 361us) server_negotiation.cc:299] Negotiation successful
0807 18:14:49.866347 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260807 18:14:49.871671 26446 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:14:49.869805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46234 (local address 127.17.110.1:45417)
0807 18:14:49.870054 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:49.870059 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:49.870188 (+ 129us) server_negotiation.cc:407] Connection header received
0807 18:14:49.870338 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:49.870342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:49.870418 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:49.870504 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:49.871387 (+ 883us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46234: BlockingRecv error: recv got EOF from 127.0.0.1:46234 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
W20260807 18:14:49.871801 17848 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126429568977-17848-0/krb5kdc/krb5cc' of type FILE
Aug 07 18:14:49 dist-test-slave-39l8 krb5kdc[25704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:49.873749 17848 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 18:14:49 dist-test-slave-39l8 krb5kdc[25704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:49.875378 17848 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 18:14:49 dist-test-slave-39l8 krb5kdc[25704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.1@KRBTEST.COM
I20260807 18:14:49.893720 26446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:49.877092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46246 (local address 127.17.110.1:45417)
0807 18:14:49.877405 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:49.877416 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:49.877820 (+ 404us) server_negotiation.cc:407] Connection header received
0807 18:14:49.878052 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:49.878058 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:49.878138 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:49.878266 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:49.879508 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:49.880336 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:49.881857 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:49.882420 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:49.882581 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:49.886889 (+ 4308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:49.886895 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:49.886898 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:49.886942 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:49.890164 (+ 3222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:49.891128 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:49.891134 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:49.891136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:49.891322 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:49.891801 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:49.891805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:49.891808 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:49.892268 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:49.892681 (+ 413us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:49.893093 (+ 412us) server_negotiation.cc:299] Negotiation successful
0807 18:14:49.893361 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260807 18:14:49.895648 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 25863
I20260807 18:14:49.910255 26446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:49.896483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46250 (local address 127.17.110.1:45417)
0807 18:14:49.896664 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:49.896670 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:49.897050 (+ 380us) server_negotiation.cc:407] Connection header received
0807 18:14:49.897180 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:49.897184 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:49.897250 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:49.897343 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:49.898361 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:49.899382 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:49.901025 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:49.901555 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:49.901720 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:49.904339 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:49.904345 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:49.904349 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:49.904388 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:49.907255 (+ 2867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:49.908035 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:49.908041 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:49.908045 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:49.908209 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:49.908650 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:49.908659 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:49.908663 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:49.908969 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:49.909294 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:49.909625 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 18:14:49.909888 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:14:49.910980 26037 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:14:50.007534 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 25863
I20260807 18:14:50.035838 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 26049
Aug 07 18:14:50 dist-test-slave-39l8 krb5kdc[25704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.2@KRBTEST.COM
I20260807 18:14:50.053010 26465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:50.036873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56820 (local address 127.17.110.2:43317)
0807 18:14:50.037495 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:50.037503 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:50.037532 (+ 29us) server_negotiation.cc:407] Connection header received
0807 18:14:50.037597 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:50.037604 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:50.037832 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:50.037968 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:50.039100 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.040219 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.041474 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.042092 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.042200 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:50.046603 (+ 4403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:50.046620 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:50.046638 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:50.046683 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:50.049970 (+ 3287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:50.050858 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:50.050866 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:50.050868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:50.051012 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:50.051390 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:50.051393 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:50.051395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:50.051765 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:50.052025 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:50.052372 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 18:14:50.052675 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":354,"threads_started":1}
I20260807 18:14:50.054427 26222 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:14:50.140628 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 26049
I20260807 18:14:50.167605 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 26244
Aug 07 18:14:50 dist-test-slave-39l8 krb5kdc[25704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.3@KRBTEST.COM
I20260807 18:14:50.186051 26468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:50.168892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38928 (local address 127.17.110.3:36861)
0807 18:14:50.169676 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:50.169684 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:50.169717 (+ 33us) server_negotiation.cc:407] Connection header received
0807 18:14:50.169786 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:50.169793 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:50.170032 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:50.170203 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:50.171478 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.172795 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.174214 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.175049 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.175186 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:50.179638 (+ 4452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:50.179656 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:50.179673 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:50.179720 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:50.183057 (+ 3337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:50.183938 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:50.183946 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:50.183948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:50.184087 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:50.184478 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:50.184482 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:50.184484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:50.184866 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:50.185107 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:50.185392 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 18:14:50.185663 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":404,"threads_started":1}
I20260807 18:14:50.187636 26423 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:14:50.276350 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 26244
I20260807 18:14:50.301231 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 25722
I20260807 18:14:50.302668 25809 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:14:50.410346 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 25722
2026-08-07T18:14:50Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4595 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:14:50 dist-test-slave-39l8 krb5kdc[26475](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:14:50 dist-test-slave-39l8 krb5kdc[26475](info): set up 2 sockets
Aug 07 18:14:50 dist-test-slave-39l8 krb5kdc[26475](info): commencing operation
krb5kdc: starting...
W20260807 18:14:51.796952 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.337s user 0.001s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:14:51 dist-test-slave-39l8 krb5kdc[26475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:14:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:14:52Z Disabled control of system clock
WARNING: no policy specified for oryx/127.17.110.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.110.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.110.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.62.keytab.
Entry for principal oryx/127.17.110.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.62.keytab.
I20260807 18:14:52.114696 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:40597
--webserver_interface=127.17.110.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.62.keytab
--principal=oryx/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:43781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:40597
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:14:52.484428 26498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:52.484772 26498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:52.484965 26498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:52.492620 26498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:14:52.492759 26498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:52.492877 26498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:52.492985 26498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:14:52.493075 26498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:14:52.504282 26498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:43781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:40597
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.62.keytab
--principal=oryx/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:40597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:52.505743 26498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:52.507294 26498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:52.520854 26503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:52.520920 26504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:52.520856 26508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:52.521988 26498 server_base.cc:1034] running on GCE node
I20260807 18:14:52.523116 26498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:52.524816 26498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:52.526140 26498 hybrid_clock.cc:648] HybridClock initialized: now 1786126492526065 us; error 75 us; skew 500 ppm
I20260807 18:14:52.526577 26498 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:52 dist-test-slave-39l8 krb5kdc[26475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126492, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:52.536351 26498 init.cc:377] Logged in from keytab as oryx/127.17.110.62@KRBTEST.COM (short username oryx)
I20260807 18:14:52.538462 26498 webserver.cc:460] Webserver started at http://127.17.110.62:46467/ using document root <none> and password file <none>
I20260807 18:14:52.539505 26498 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:52.539700 26498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:52.540118 26498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:52.543888 26498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2"
format_stamp: "Formatted at 2026-08-07 18:14:52 on dist-test-slave-39l8"
server_key: "cc5a8fb9138b4856131cdb19cb9f4fcb"
server_key_iv: "7c82af720aa1404c2831e6fd7d2a2ea3"
server_key_version: "encryptionkey@0"
I20260807 18:14:52.544840 26498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2"
format_stamp: "Formatted at 2026-08-07 18:14:52 on dist-test-slave-39l8"
server_key: "cc5a8fb9138b4856131cdb19cb9f4fcb"
server_key_iv: "7c82af720aa1404c2831e6fd7d2a2ea3"
server_key_version: "encryptionkey@0"
I20260807 18:14:52.551474 26498 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 18:14:52.556048 26517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:52.557801 26498 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.004s
I20260807 18:14:52.558120 26498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2"
format_stamp: "Formatted at 2026-08-07 18:14:52 on dist-test-slave-39l8"
server_key: "cc5a8fb9138b4856131cdb19cb9f4fcb"
server_key_iv: "7c82af720aa1404c2831e6fd7d2a2ea3"
server_key_version: "encryptionkey@0"
I20260807 18:14:52.558408 26498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:52.577782 26498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:52.583269 26498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:52.583632 26498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:52.605387 26498 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:40597
I20260807 18:14:52.605424 26594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:40597 every 8 connection(s)
I20260807 18:14:52.606935 26498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:14:52.611090 26599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:52.615561 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 26498
I20260807 18:14:52.615818 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:14:52.616490 17848 external_mini_cluster.cc:1428] Setting key e670a59339a1627c3936f133e1b565e1
I20260807 18:14:52.624722 26599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2: Bootstrap starting.
I20260807 18:14:52.628746 26599 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:52.630231 26599 log.cc:824] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:14:52 dist-test-slave-39l8 krb5kdc[26475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM
I20260807 18:14:52.633965 26599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2: No bootstrap required, opened a new log
I20260807 18:14:52.644897 26603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:52.618871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58322 (local address 127.17.110.62:40597)
0807 18:14:52.619795 (+ 924us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:52.619813 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:52.619868 (+ 55us) server_negotiation.cc:407] Connection header received
0807 18:14:52.622619 (+ 2751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:52.622660 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:52.623292 (+ 632us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:52.623791 (+ 499us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:52.625402 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:52.627569 (+ 2167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:52.628905 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:52.630046 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:52.630262 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:52.635132 (+ 4870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:52.635144 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:52.635167 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:52.635220 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:52.639727 (+ 4507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:52.640562 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:52.640573 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:52.640588 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:52.640827 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:52.641209 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:52.641223 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:52.641226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:52.642093 (+ 867us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:52.642452 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:52.642813 (+ 361us) server_negotiation.cc:299] Negotiation successful
0807 18:14:52.643347 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":385,"threads_started":1}
I20260807 18:14:52.645746 26599 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } }
I20260807 18:14:52.646219 26599 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:52.646351 26599 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce9e1ae9d6354f9d9b7020b5e73f2ef2, State: Initialized, Role: FOLLOWER
I20260807 18:14:52.647106 26599 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } }
I20260807 18:14:52.647401 26599 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:14:52.647560 26599 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:14:52.647769 26599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:52.650220 26599 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } }
I20260807 18:14:52.650871 26599 leader_election.cc:302] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce9e1ae9d6354f9d9b7020b5e73f2ef2; no voters:
I20260807 18:14:52.651468 26599 leader_election.cc:288] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:14:52.651666 26605 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:14:52.652573 26605 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [term 1 LEADER]: Becoming Leader. State: Replica: ce9e1ae9d6354f9d9b7020b5e73f2ef2, State: Running, Role: LEADER
I20260807 18:14:52.653180 26605 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } }
I20260807 18:14:52.653947 26599 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:14:52.658038 26606 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } }
I20260807 18:14:52.658536 26606 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:52.659462 26608 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce9e1ae9d6354f9d9b7020b5e73f2ef2. Latest consensus state: current_term: 1 leader_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } }
I20260807 18:14:52.659924 26608 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:52.662818 26622 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:14:52.668617 26622 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:14:52.679277 26622 catalog_manager.cc:1347] Generated new cluster ID: 2749ae813a664a0f81bd97be8eb21721
I20260807 18:14:52.679440 26622 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:14:52.719005 26622 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:14:52.720556 26622 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:14:52.729621 26622 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2: Generated new TSK 0
I20260807 18:14:52.730563 26622 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.17.110.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.110.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.110.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.1.keytab.
Entry for principal oryx/127.17.110.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.1.keytab.
I20260807 18:14:52.798645 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.1:0
--local_ip_for_outbound_sockets=127.17.110.1
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:40597
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.1.keytab
--principal=oryx/127.17.110.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:43781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:14:53.169051 26639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:53.169440 26639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:53.169620 26639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:53.177520 26639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:53.177706 26639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:53.177886 26639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.1
I20260807 18:14:53.188417 26639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:43781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.1.keytab
--principal=oryx/127.17.110.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:40597
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.1
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:53.189894 26639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:53.191375 26639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:53.205765 26646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:53.205802 26653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:53.205860 26648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:53.207103 26639 server_base.cc:1034] running on GCE node
I20260807 18:14:53.207831 26639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:53.208842 26639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:53.210071 26639 hybrid_clock.cc:648] HybridClock initialized: now 1786126493210012 us; error 52 us; skew 500 ppm
I20260807 18:14:53.210497 26639 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:53 dist-test-slave-39l8 krb5kdc[26475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126493, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:53.214181 26639 init.cc:377] Logged in from keytab as oryx/127.17.110.1@KRBTEST.COM (short username oryx)
I20260807 18:14:53.215873 26639 webserver.cc:460] Webserver started at http://127.17.110.1:40233/ using document root <none> and password file <none>
I20260807 18:14:53.216657 26639 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:53.216823 26639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:53.217171 26639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:53.220490 26639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-0/data/instance:
uuid: "d6fda665629f44b69b25453b35fb9490"
format_stamp: "Formatted at 2026-08-07 18:14:53 on dist-test-slave-39l8"
server_key: "875295331fd56895dfe5cdd9963fe0ab"
server_key_iv: "4419ff81956ad9987e3aa58d8673a6f6"
server_key_version: "encryptionkey@0"
I20260807 18:14:53.221415 26639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance:
uuid: "d6fda665629f44b69b25453b35fb9490"
format_stamp: "Formatted at 2026-08-07 18:14:53 on dist-test-slave-39l8"
server_key: "875295331fd56895dfe5cdd9963fe0ab"
server_key_iv: "4419ff81956ad9987e3aa58d8673a6f6"
server_key_version: "encryptionkey@0"
I20260807 18:14:53.227526 26639 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 18:14:53.231668 26660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:53.233470 26639 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260807 18:14:53.233698 26639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-0/wal
uuid: "d6fda665629f44b69b25453b35fb9490"
format_stamp: "Formatted at 2026-08-07 18:14:53 on dist-test-slave-39l8"
server_key: "875295331fd56895dfe5cdd9963fe0ab"
server_key_iv: "4419ff81956ad9987e3aa58d8673a6f6"
server_key_version: "encryptionkey@0"
I20260807 18:14:53.233937 26639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:53.255882 26639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:53.261148 26639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:53.261498 26639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:53.262781 26639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:14:53.264396 26639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:14:53.264479 26639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:53.264621 26639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:14:53.264688 26639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:53.302898 26639 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.1:33421
I20260807 18:14:53.302956 26833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.1:33421 every 8 connection(s)
I20260807 18:14:53.304422 26639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
I20260807 18:14:53.308954 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 26639
I20260807 18:14:53.309281 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance
I20260807 18:14:53.310334 17848 external_mini_cluster.cc:1428] Setting key ad78bf1935ff42bff5cfe7f3bc15ca81
Aug 07 18:14:53 dist-test-slave-39l8 krb5kdc[26475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126493, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM
I20260807 18:14:53.334005 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:53.307430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:52089 (local address 127.17.110.62:40597)
0807 18:14:53.307938 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:53.307944 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:53.309235 (+ 1291us) server_negotiation.cc:407] Connection header received
0807 18:14:53.312262 (+ 3027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:53.312268 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:53.312360 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:53.312498 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:53.315349 (+ 2851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:53.316407 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:53.319486 (+ 3079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:53.319958 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:53.320106 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:53.325616 (+ 5510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:53.325627 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:53.325635 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:53.325691 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:53.329772 (+ 4081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:53.330835 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:53.330844 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:53.330850 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:53.331023 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:53.331573 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:53.331581 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:53.331587 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:53.332092 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:53.332266 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:53.333171 (+ 905us) server_negotiation.cc:299] Negotiation successful
0807 18:14:53.333495 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":267,"threads_started":1}
I20260807 18:14:53.334864 26838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:53.307989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.17.110.1:52089)
0807 18:14:53.309005 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:53.309064 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:53.311905 (+ 2841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:53.313002 (+ 1097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:53.313018 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:53.314028 (+ 1010us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:53.315091 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:53.315130 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:53.316615 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:53.316624 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:53.319236 (+ 2612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:53.319262 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:53.320290 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:53.320303 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:53.320712 (+ 409us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:53.321681 (+ 969us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:53.321733 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:53.325248 (+ 3515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:53.330003 (+ 4755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:53.330013 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:53.330033 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:53.330610 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:53.331168 (+ 558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:53.331173 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:53.331177 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:53.331424 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:53.332241 (+ 817us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:53.332249 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:53.332652 (+ 403us) client_negotiation.cc:769] Sending connection context
0807 18:14:53.332950 (+ 298us) client_negotiation.cc:240] Negotiation successful
0807 18:14:53.333443 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"thread_start_us":367,"threads_started":1}
I20260807 18:14:53.336963 26834 heartbeater.cc:347] Connected to a master server at 127.17.110.62:40597
I20260807 18:14:53.337352 26834 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:53.338439 26834 heartbeater.cc:511] Master 127.17.110.62:40597 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.17.110.2@KRBTEST.COM; defaulting to no policy
I20260807 18:14:53.341184 26546 ts_manager.cc:194] Registered new tserver with Master: d6fda665629f44b69b25453b35fb9490 (127.17.110.1:33421)
I20260807 18:14:53.343047 26546 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.110.1@KRBTEST.COM'} at 127.17.110.1:52089
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.110.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.110.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.2.keytab.
Entry for principal oryx/127.17.110.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.2.keytab.
I20260807 18:14:53.378439 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.2:0
--local_ip_for_outbound_sockets=127.17.110.2
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:40597
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.2.keytab
--principal=oryx/127.17.110.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:43781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:14:53.746775 26843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:53.747102 26843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:53.747303 26843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:53.754853 26843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:53.754969 26843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:53.755180 26843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.2
I20260807 18:14:53.765619 26843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:43781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.2.keytab
--principal=oryx/127.17.110.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:40597
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.2
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:53.767014 26843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:53.768356 26843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:53.782614 26850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:53.782917 26851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:53.783294 26856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:53.784061 26843 server_base.cc:1034] running on GCE node
I20260807 18:14:53.784868 26843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:53.785935 26843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:53.787175 26843 hybrid_clock.cc:648] HybridClock initialized: now 1786126493787107 us; error 44 us; skew 500 ppm
I20260807 18:14:53.787515 26843 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:53 dist-test-slave-39l8 krb5kdc[26475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126493, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:53.791172 26843 init.cc:377] Logged in from keytab as oryx/127.17.110.2@KRBTEST.COM (short username oryx)
I20260807 18:14:53.792806 26843 webserver.cc:460] Webserver started at http://127.17.110.2:33255/ using document root <none> and password file <none>
I20260807 18:14:53.793603 26843 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:53.793720 26843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:53.794014 26843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:53.797219 26843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-1/data/instance:
uuid: "480a54d1b82d42acb4428942491d0910"
format_stamp: "Formatted at 2026-08-07 18:14:53 on dist-test-slave-39l8"
server_key: "5678041c1c6867cf3e718ce526290c6c"
server_key_iv: "109cf82c5eae88b673e79e24da40be1a"
server_key_version: "encryptionkey@0"
I20260807 18:14:53.798035 26843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance:
uuid: "480a54d1b82d42acb4428942491d0910"
format_stamp: "Formatted at 2026-08-07 18:14:53 on dist-test-slave-39l8"
server_key: "5678041c1c6867cf3e718ce526290c6c"
server_key_iv: "109cf82c5eae88b673e79e24da40be1a"
server_key_version: "encryptionkey@0"
I20260807 18:14:53.804028 26843 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 18:14:53.808176 26864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:53.809926 26843 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:14:53.810168 26843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-1/wal
uuid: "480a54d1b82d42acb4428942491d0910"
format_stamp: "Formatted at 2026-08-07 18:14:53 on dist-test-slave-39l8"
server_key: "5678041c1c6867cf3e718ce526290c6c"
server_key_iv: "109cf82c5eae88b673e79e24da40be1a"
server_key_version: "encryptionkey@0"
I20260807 18:14:53.810498 26843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:53.828739 26843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:53.833254 26843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:53.833597 26843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:53.834815 26843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:14:53.836416 26843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:14:53.836530 26843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:53.836700 26843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:14:53.836776 26843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:53.873746 26843 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.2:40847
I20260807 18:14:53.873795 27009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.2:40847 every 8 connection(s)
I20260807 18:14:53.875159 26843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
I20260807 18:14:53.876718 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 26843
I20260807 18:14:53.876924 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance
I20260807 18:14:53.877436 17848 external_mini_cluster.cc:1428] Setting key 7c522e3636424de5145ba6cf0c032646
Aug 07 18:14:53 dist-test-slave-39l8 krb5kdc[26475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126493, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM
I20260807 18:14:53.902937 27014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:53.878127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:47623 (local address 127.17.110.62:40597)
0807 18:14:53.878645 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:53.878650 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:53.879952 (+ 1302us) server_negotiation.cc:407] Connection header received
0807 18:14:53.883223 (+ 3271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:53.883228 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:53.883321 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:53.883435 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:53.885915 (+ 2480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:53.886919 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:53.889756 (+ 2837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:53.890522 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:53.890648 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:53.895776 (+ 5128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:53.895782 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:53.895786 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:53.895831 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:53.898996 (+ 3165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:53.899961 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:53.899975 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:53.899978 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:53.900133 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:53.900684 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:53.900703 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:53.900706 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:53.901102 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:53.901336 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:53.902156 (+ 820us) server_negotiation.cc:299] Negotiation successful
0807 18:14:53.902428 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":282,"threads_started":1}
I20260807 18:14:53.904155 27015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:53.878764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.17.110.2:47623)
0807 18:14:53.879731 (+ 967us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:53.879799 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:53.882811 (+ 3012us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:53.883780 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:53.883797 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:53.884691 (+ 894us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:53.885621 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:53.885654 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:53.887140 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:53.887148 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:53.889503 (+ 2355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:53.889525 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:53.890850 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:53.890860 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:53.891252 (+ 392us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:53.892191 (+ 939us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:53.892232 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:53.895420 (+ 3188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:53.899220 (+ 3800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:53.899235 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:53.899260 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:53.899793 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:53.900283 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:53.900289 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:53.900294 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:53.900543 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:53.901279 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:53.901289 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:53.901739 (+ 450us) client_negotiation.cc:769] Sending connection context
0807 18:14:53.902065 (+ 326us) client_negotiation.cc:240] Negotiation successful
0807 18:14:53.902783 (+ 718us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":351,"threads_started":1}
WARNING: no policy specified for oryx/127.17.110.3@KRBTEST.COM; defaulting to no policy
I20260807 18:14:53.905967 27010 heartbeater.cc:347] Connected to a master server at 127.17.110.62:40597
I20260807 18:14:53.906338 27010 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:53.907302 27010 heartbeater.cc:511] Master 127.17.110.62:40597 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.110.3@KRBTEST.COM" created.
I20260807 18:14:53.909266 26546 ts_manager.cc:194] Registered new tserver with Master: 480a54d1b82d42acb4428942491d0910 (127.17.110.2:40847)
I20260807 18:14:53.910183 26546 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.110.2@KRBTEST.COM'} at 127.17.110.2:47623
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.110.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.3.keytab.
Entry for principal oryx/127.17.110.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.3.keytab.
I20260807 18:14:53.941905 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.3:0
--local_ip_for_outbound_sockets=127.17.110.3
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:40597
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.3.keytab
--principal=oryx/127.17.110.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:43781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:14:54.308889 27021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:54.309252 27021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:54.309428 27021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:54.317548 27021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:54.317745 27021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:54.317930 27021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.3
I20260807 18:14:54.328572 27021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:43781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.3.keytab
--principal=oryx/127.17.110.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:40597
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.3
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:54.330034 27021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:54.331534 27021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:54.346607 27028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:54.346822 26834 heartbeater.cc:503] Master 127.17.110.62:40597 was elected leader, sending a full tablet report...
W20260807 18:14:54.346515 27032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:54.346549 27027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:54.347544 27021 server_base.cc:1034] running on GCE node
I20260807 18:14:54.348291 27021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:54.349524 27021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:54.350803 27021 hybrid_clock.cc:648] HybridClock initialized: now 1786126494350740 us; error 48 us; skew 500 ppm
I20260807 18:14:54.351168 27021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:54 dist-test-slave-39l8 krb5kdc[26475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126494, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:54.355068 27021 init.cc:377] Logged in from keytab as oryx/127.17.110.3@KRBTEST.COM (short username oryx)
I20260807 18:14:54.356844 27021 webserver.cc:460] Webserver started at http://127.17.110.3:34993/ using document root <none> and password file <none>
I20260807 18:14:54.357622 27021 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:54.357779 27021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:54.358173 27021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:54.361586 27021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-2/data/instance:
uuid: "25c40cc9663e47c8b30a95e33060db88"
format_stamp: "Formatted at 2026-08-07 18:14:54 on dist-test-slave-39l8"
server_key: "e6aecb5c7dd87543d7a05d55c302f74e"
server_key_iv: "9f13677d63f8d19b322f91a014714f2d"
server_key_version: "encryptionkey@0"
I20260807 18:14:54.362475 27021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance:
uuid: "25c40cc9663e47c8b30a95e33060db88"
format_stamp: "Formatted at 2026-08-07 18:14:54 on dist-test-slave-39l8"
server_key: "e6aecb5c7dd87543d7a05d55c302f74e"
server_key_iv: "9f13677d63f8d19b322f91a014714f2d"
server_key_version: "encryptionkey@0"
I20260807 18:14:54.368629 27021 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260807 18:14:54.372691 27040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:54.374437 27021 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:14:54.374667 27021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-2/wal
uuid: "25c40cc9663e47c8b30a95e33060db88"
format_stamp: "Formatted at 2026-08-07 18:14:54 on dist-test-slave-39l8"
server_key: "e6aecb5c7dd87543d7a05d55c302f74e"
server_key_iv: "9f13677d63f8d19b322f91a014714f2d"
server_key_version: "encryptionkey@0"
I20260807 18:14:54.375010 27021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:54.395524 27021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:54.400167 27021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:54.400509 27021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:54.401584 27021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:14:54.403258 27021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:14:54.403385 27021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:54.403555 27021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:14:54.403632 27021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:54.441488 27021 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.3:37809
I20260807 18:14:54.441548 27193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.3:37809 every 8 connection(s)
I20260807 18:14:54.442929 27021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
I20260807 18:14:54.451401 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 27021
I20260807 18:14:54.451642 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance
I20260807 18:14:54.452159 17848 external_mini_cluster.cc:1428] Setting key cc84e17657f25f69fd8a777fe928dd64
Aug 07 18:14:54 dist-test-slave-39l8 krb5kdc[26475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126494, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM
I20260807 18:14:54.467053 27198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:54.445859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:42673 (local address 127.17.110.62:40597)
0807 18:14:54.446415 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:54.446421 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:54.447544 (+ 1123us) server_negotiation.cc:407] Connection header received
0807 18:14:54.449688 (+ 2144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:54.449695 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:54.449827 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:54.449975 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:54.451975 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:54.452925 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:54.454910 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:54.455499 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:54.455660 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:54.459760 (+ 4100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:54.459766 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:54.459769 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:54.459817 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:54.463216 (+ 3399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:54.464186 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:54.464211 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:54.464215 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:54.464359 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:54.464916 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:54.464921 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:54.464923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:54.465247 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:54.465442 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:54.466306 (+ 864us) server_negotiation.cc:299] Negotiation successful
0807 18:14:54.466573 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":289,"threads_started":1}
I20260807 18:14:54.468394 27200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:54.446444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.17.110.3:42673)
0807 18:14:54.447353 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:54.447397 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:54.449339 (+ 1942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:54.450285 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:54.450295 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:54.451005 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:54.451755 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:54.451786 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:54.453126 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:54.453132 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:54.454658 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:54.454673 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:54.455802 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:54.455813 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:54.456079 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:54.456872 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:54.456905 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:54.459448 (+ 2543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:54.463416 (+ 3968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:54.463428 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:54.463452 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:54.464013 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:54.464502 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:54.464508 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:54.464512 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:54.464762 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:54.465408 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:54.465418 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:54.465852 (+ 434us) client_negotiation.cc:769] Sending connection context
0807 18:14:54.466166 (+ 314us) client_negotiation.cc:240] Negotiation successful
0807 18:14:54.466861 (+ 695us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":376,"threads_started":1}
I20260807 18:14:54.469847 27194 heartbeater.cc:347] Connected to a master server at 127.17.110.62:40597
I20260807 18:14:54.470201 27194 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:54.471122 27194 heartbeater.cc:511] Master 127.17.110.62:40597 requested a full tablet report, sending...
I20260807 18:14:54.472785 26546 ts_manager.cc:194] Registered new tserver with Master: 25c40cc9663e47c8b30a95e33060db88 (127.17.110.3:37809)
I20260807 18:14:54.473639 26546 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.110.3@KRBTEST.COM'} at 127.17.110.3:42673
I20260807 18:14:54.481091 17848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.17.110.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.110.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.110.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.61.keytab.
Entry for principal oryx/127.17.110.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.61.keytab.
I20260807 18:14:54.541949 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.61:37317
--webserver_interface=127.17.110.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.61.keytab
--principal=oryx/127.17.110.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:43781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:40597,127.17.110.61:37317
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:14:54.914497 27010 heartbeater.cc:503] Master 127.17.110.62:40597 was elected leader, sending a full tablet report...
W20260807 18:14:54.919991 27204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:54.920415 27204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:54.920598 27204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:54.928964 27204 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:14:54.929155 27204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:54.929283 27204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:54.929391 27204 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:14:54.929483 27204 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:14:54.940250 27204 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:43781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:40597,127.17.110.61:37317
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.61.keytab
--principal=oryx/127.17.110.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.61:37317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.110.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
W20260807 18:14:54.940712 27204 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.17.110.62:40597,127.17.110.61:37317'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 18:14:54.941875 27204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:54.943341 27204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:54.956324 27211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:54.956246 27217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:54.956302 27213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:54.957535 27204 server_base.cc:1034] running on GCE node
I20260807 18:14:54.958482 27204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:54.960152 27204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:54.961417 27204 hybrid_clock.cc:648] HybridClock initialized: now 1786126494961361 us; error 40 us; skew 500 ppm
I20260807 18:14:54.961843 27204 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:54 dist-test-slave-39l8 krb5kdc[26475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126494, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:54.965611 27204 init.cc:377] Logged in from keytab as oryx/127.17.110.61@KRBTEST.COM (short username oryx)
I20260807 18:14:54.967293 27204 webserver.cc:460] Webserver started at http://127.17.110.61:35243/ using document root <none> and password file <none>
I20260807 18:14:54.968168 27204 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:54.968277 27204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:54.968578 27204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:54.971887 27204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-1/data/instance:
uuid: "94d47e0385a840749cd3676abfda32c0"
format_stamp: "Formatted at 2026-08-07 18:14:54 on dist-test-slave-39l8"
server_key: "81c2f499b8e8c9afce97b5fa2b7ed3f7"
server_key_iv: "2e5f156eb83314856340d9a50261b826"
server_key_version: "encryptionkey@0"
I20260807 18:14:54.972728 27204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-1/wal/instance:
uuid: "94d47e0385a840749cd3676abfda32c0"
format_stamp: "Formatted at 2026-08-07 18:14:54 on dist-test-slave-39l8"
server_key: "81c2f499b8e8c9afce97b5fa2b7ed3f7"
server_key_iv: "2e5f156eb83314856340d9a50261b826"
server_key_version: "encryptionkey@0"
I20260807 18:14:54.979888 27204 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260807 18:14:54.984289 27226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:54.985963 27204 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260807 18:14:54.986151 27204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-1/wal
uuid: "94d47e0385a840749cd3676abfda32c0"
format_stamp: "Formatted at 2026-08-07 18:14:54 on dist-test-slave-39l8"
server_key: "81c2f499b8e8c9afce97b5fa2b7ed3f7"
server_key_iv: "2e5f156eb83314856340d9a50261b826"
server_key_version: "encryptionkey@0"
I20260807 18:14:54.986361 27204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:55.008495 27204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:55.013577 27204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:55.013897 27204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:55.035591 27204 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.61:37317
I20260807 18:14:55.035629 27294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.61:37317 every 8 connection(s)
I20260807 18:14:55.037143 27204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-1/data/info.pb
I20260807 18:14:55.041379 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 27204
I20260807 18:14:55.041414 27296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:55.050863 27296 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:14:55 dist-test-slave-39l8 krb5kdc[26475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.110.61@KRBTEST.COM
Aug 07 18:14:55 dist-test-slave-39l8 krb5kdc[26475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126494, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.61@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM
I20260807 18:14:55.067870 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:55.043845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47398 (local address 127.17.110.61:37317)
0807 18:14:55.044762 (+ 917us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:55.044776 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:55.044819 (+ 43us) server_negotiation.cc:407] Connection header received
0807 18:14:55.046684 (+ 1865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:55.046766 (+ 82us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:55.047214 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:55.047688 (+ 474us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:55.049329 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.050663 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.052181 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.052914 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.053150 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:55.058661 (+ 5511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:55.058669 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:55.058681 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:55.058783 (+ 102us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:55.063024 (+ 4241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:55.063918 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:55.063930 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:55.063943 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:55.064184 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:55.064705 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:55.064709 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:55.064711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:55.065529 (+ 818us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:55.065854 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:55.066286 (+ 432us) server_negotiation.cc:299] Negotiation successful
0807 18:14:55.066682 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":420,"threads_started":1}
I20260807 18:14:55.072854 27301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:55.053718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58326 (local address 127.17.110.62:40597)
0807 18:14:55.054517 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:55.054526 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:55.055656 (+ 1130us) server_negotiation.cc:407] Connection header received
0807 18:14:55.055860 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:55.055901 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:55.055975 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:55.056077 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:55.057663 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.058851 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.061234 (+ 2383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.061716 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.061857 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:55.066107 (+ 4250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:55.066114 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:55.066119 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:55.066176 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:55.069471 (+ 3295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:55.070316 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:55.070324 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:55.070328 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:55.070484 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:55.070986 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:55.070991 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:55.071000 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:55.071516 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:55.071695 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:55.072219 (+ 524us) server_negotiation.cc:299] Negotiation successful
0807 18:14:55.072468 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":442,"threads_started":1}
I20260807 18:14:55.072903 27302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:55.053715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.0.0.1:58326)
0807 18:14:55.055471 (+ 1756us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:55.055528 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:55.055724 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:55.056205 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:55.056213 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:55.057097 (+ 884us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:55.057483 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:55.057495 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.058994 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.059001 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:55.061016 (+ 2015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:55.061038 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.061935 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.061941 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:55.062138 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:55.062900 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:55.062931 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:55.065847 (+ 2916us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:55.069632 (+ 3785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:55.069641 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:55.069647 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:55.070164 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:55.070608 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:55.070611 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:55.070614 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:55.070856 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:55.071659 (+ 803us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:55.071667 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:55.071879 (+ 212us) client_negotiation.cc:769] Sending connection context
0807 18:14:55.072094 (+ 215us) client_negotiation.cc:240] Negotiation successful
0807 18:14:55.072466 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1428,"thread_start_us":1199,"threads_started":1}
I20260807 18:14:55.074976 27296 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:14:55 dist-test-slave-39l8 krb5kdc[26475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126494, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.61@KRBTEST.COM for oryx/127.17.110.61@KRBTEST.COM
I20260807 18:14:55.090045 27302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:55.075889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:37317 (local address 127.0.0.1:47412)
0807 18:14:55.076214 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:55.076241 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:55.076384 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:55.076725 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:55.076730 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:55.077177 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:55.077327 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:55.077336 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.078470 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.078476 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:55.079601 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:55.079644 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.080541 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.080547 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:55.080733 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:55.081427 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:55.081447 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:55.083955 (+ 2508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:55.087463 (+ 3508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:55.087468 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:55.087471 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:55.087965 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:55.088391 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:55.088395 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:55.088398 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:55.088550 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:55.089216 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:55.089220 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:55.089365 (+ 145us) client_negotiation.cc:769] Sending connection context
0807 18:14:55.089497 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 18:14:55.089776 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260807 18:14:55.090440 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:55.076157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47412 (local address 127.17.110.61:37317)
0807 18:14:55.076339 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:55.076345 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:55.076365 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:14:55.076513 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:55.076519 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:55.076594 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:55.076710 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:55.077483 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.078275 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.079825 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.080329 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.080459 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:55.084200 (+ 3741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:55.084205 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:55.084209 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:55.084243 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:55.087269 (+ 3026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:55.088123 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:55.088128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:55.088130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:55.088260 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:55.088726 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:55.088730 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:55.088732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:55.089059 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:55.089210 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:55.089612 (+ 402us) server_negotiation.cc:299] Negotiation successful
0807 18:14:55.089952 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":77,"server-negotiator.queue_time_us":65}
I20260807 18:14:55.095814 27296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0: Bootstrap starting.
I20260807 18:14:55.099416 27296 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:55.100646 27296 log.cc:824] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0: Log is configured to *not* fsync() on all Append() calls
I20260807 18:14:55.104039 27296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0: No bootstrap required, opened a new log
I20260807 18:14:55.112028 27296 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } }
I20260807 18:14:55.112501 27296 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:55.112632 27296 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94d47e0385a840749cd3676abfda32c0, State: Initialized, Role: FOLLOWER
I20260807 18:14:55.113373 27296 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } }
I20260807 18:14:55.114650 27306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } } }
I20260807 18:14:55.115214 27306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:14:55.115888 27296 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 18:14:55.126763 27319 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:14:55.127012 27319 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:14:55.129835 27204 master_runner.cc:428] Detected that this master 94d47e0385a840749cd3676abfda32c0 is joining an existing cluster
I20260807 18:14:55.129974 27204 master_runner.cc:432] Initiating AddMaster RPC to add 127.17.110.61:37317
I20260807 18:14:55.156176 27306 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:14:55.156548 27306 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } }
WARNING: no policy specified for oryx/127.17.110.60@KRBTEST.COM; defaulting to no policy
I20260807 18:14:55.158249 27306 leader_election.cc:288] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce9e1ae9d6354f9d9b7020b5e73f2ef2 (127.17.110.62:40597)
I20260807 18:14:55.159511 26563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "94d47e0385a840749cd3676abfda32c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" is_pre_election: true
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.110.60@KRBTEST.COM" created.
I20260807 18:14:55.161430 27227 leader_election.cc:302] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 94d47e0385a840749cd3676abfda32c0; no voters: ce9e1ae9d6354f9d9b7020b5e73f2ef2
I20260807 18:14:55.162319 27306 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 18:14:55.163961 27301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:55.138496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58336 (local address 127.17.110.62:40597)
0807 18:14:55.138809 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:55.138816 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:55.139575 (+ 759us) server_negotiation.cc:407] Connection header received
0807 18:14:55.139820 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:55.139829 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:55.139941 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:55.140022 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:55.142341 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.143592 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.146050 (+ 2458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.146942 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.147066 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:55.151450 (+ 4384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:55.151458 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:55.151463 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:55.151512 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:55.155618 (+ 4106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:55.160765 (+ 5147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:55.160774 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:55.160778 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:55.160993 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:55.161713 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:55.161721 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:55.161727 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:55.162144 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:55.162304 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:55.163281 (+ 977us) server_negotiation.cc:299] Negotiation successful
0807 18:14:55.163576 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260807 18:14:55.165319 27330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:55.138530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.0.0.1:58336)
0807 18:14:55.139429 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:55.139450 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:55.139614 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:55.140278 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:55.140285 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:55.141107 (+ 822us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:55.142083 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:55.142103 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.143821 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.143829 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:55.145839 (+ 2010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:55.145860 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.147439 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.147448 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:55.147687 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:55.148514 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:55.148543 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:55.151125 (+ 2582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:55.159234 (+ 8109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:55.159240 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:55.159245 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:55.160592 (+ 1347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:55.161206 (+ 614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:55.161216 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:55.161220 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:55.161545 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:55.162388 (+ 843us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:55.162392 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:55.162913 (+ 521us) client_negotiation.cc:769] Sending connection context
0807 18:14:55.164379 (+ 1466us) client_negotiation.cc:240] Negotiation successful
0807 18:14:55.164777 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"thread_start_us":535,"threads_started":1}
I20260807 18:14:55.175292 27331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:55.139654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:37317 (local address 127.0.0.1:47418)
0807 18:14:55.151517 (+ 11863us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:55.151533 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:55.151692 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:55.152271 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:55.152278 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:55.153052 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:55.153434 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:55.153445 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.155499 (+ 2054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.155505 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:55.157269 (+ 1764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:55.157288 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.158926 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.158932 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:55.159153 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:55.159886 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:55.159913 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:55.163225 (+ 3312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:55.169181 (+ 5956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:55.169189 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:55.169195 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:55.169837 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:55.170298 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:55.170305 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:55.170311 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:55.170498 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:55.172623 (+ 2125us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:55.172628 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:55.174204 (+ 1576us) client_negotiation.cc:769] Sending connection context
0807 18:14:55.174402 (+ 198us) client_negotiation.cc:240] Negotiation successful
0807 18:14:55.174788 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11724,"thread_start_us":11395,"threads_started":1}
I20260807 18:14:55.176326 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:55.139968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47418 (local address 127.17.110.61:37317)
0807 18:14:55.140196 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:55.140201 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:55.151626 (+ 11425us) server_negotiation.cc:407] Connection header received
0807 18:14:55.151830 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:55.151838 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:55.151918 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:55.152001 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:55.153903 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.155255 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.157454 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.158335 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.158450 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:55.163446 (+ 4996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:55.163460 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:55.163467 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:55.163518 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:55.168991 (+ 5473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:55.169971 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:55.169979 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:55.169986 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:55.170162 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:55.170611 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:55.170618 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:55.170624 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:55.171090 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:55.172523 (+ 1433us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:55.175452 (+ 2929us) server_negotiation.cc:299] Negotiation successful
0807 18:14:55.175773 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
Aug 07 18:14:55 dist-test-slave-39l8 krb5kdc[26475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126492, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.62@KRBTEST.COM for oryx/127.17.110.61@KRBTEST.COM
I20260807 18:14:55.203223 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:55.173241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47422 (local address 127.17.110.61:37317)
0807 18:14:55.180406 (+ 7165us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:55.180415 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:55.180440 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:14:55.180556 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:55.180564 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:55.180684 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:55.180820 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:55.183213 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.184801 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.187977 (+ 3176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.189090 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.189291 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:55.195237 (+ 5946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:55.195245 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:55.195251 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:55.195301 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:55.199085 (+ 3784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:55.200362 (+ 1277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:55.200367 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:55.200370 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:55.200556 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:55.201129 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:55.201135 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:55.201139 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:55.201588 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:55.201822 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:55.202413 (+ 591us) server_negotiation.cc:299] Negotiation successful
0807 18:14:55.202697 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6997,"thread_start_us":17047,"threads_started":1}
I20260807 18:14:55.203290 27336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:55.173242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:37317 (local address 127.0.0.1:47422)
0807 18:14:55.176401 (+ 3159us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:55.176446 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:55.176622 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:55.181202 (+ 4580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:55.181214 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:55.182448 (+ 1234us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:55.182920 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:55.182943 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.185050 (+ 2107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.185059 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:55.187661 (+ 2602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:55.187694 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.190040 (+ 2346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.190051 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:55.190383 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:55.191294 (+ 911us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:55.191335 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:55.194866 (+ 3531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:55.199303 (+ 4437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:55.199327 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:55.199339 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:55.200092 (+ 753us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:55.200716 (+ 624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:55.200722 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:55.200727 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:55.200943 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:55.201756 (+ 813us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:55.201767 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:55.202009 (+ 242us) client_negotiation.cc:769] Sending connection context
0807 18:14:55.202275 (+ 266us) client_negotiation.cc:240] Negotiation successful
0807 18:14:55.202749 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2884,"thread_start_us":3445,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.110.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.60.keytab.
Entry for principal oryx/127.17.110.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.60.keytab.
I20260807 18:14:55.205838 26546 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.17.110.61:37317: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "94d47e0385a840749cd3676abfda32c0"
member_type: NON_VOTER
last_known_addr {
host: "127.17.110.61"
port: 37317
}
attrs {
promote: true
}
}
dest_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2"
cas_config_opid_index: -1
I20260807 18:14:55.207836 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.60:34151
--webserver_interface=127.17.110.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.60.keytab
--principal=oryx/127.17.110.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:43781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:40597,127.17.110.61:37317,127.17.110.60:34151
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:14:55.207690 26546 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: NON_VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: true } }
I20260807 18:14:55.217461 27263 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:55.217083 27341 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 94d47e0385a840749cd3676abfda32c0 (127.17.110.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: NON_VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: true } } }
I20260807 18:14:55.221644 27263 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [term 1 FOLLOWER]: Refusing update from remote peer ce9e1ae9d6354f9d9b7020b5e73f2ef2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 18:14:55.222754 27341 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.17.110.61:37317
I20260807 18:14:55.223482 27340 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: NON_VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: true } } }
I20260807 18:14:55.223934 27204 master.cc:537] Master@127.17.110.61:37317 shutting down...
I20260807 18:14:55.224032 27340 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:55.224888 27341 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: NON_VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:14:55.242255 27204 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:14:55.242897 27204 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:14:55.243201 27204 tablet_replica.cc:330] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0: stopping tablet replica
W20260807 18:14:55.243605 26518 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260807 18:14:55.246270 26518 consensus_peers.cc:487] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 -> Peer 94d47e0385a840749cd3676abfda32c0 (127.17.110.61:37317): Couldn't send request to peer 94d47e0385a840749cd3676abfda32c0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 18:14:55.251634 27204 master.cc:552] Master@127.17.110.61:37317 shutdown complete.
I20260807 18:14:55.251802 27204 master_runner.cc:305] Clearing existing system tablet
I20260807 18:14:55.252480 27204 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:55.252609 27204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:55.255962 27204 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.004s
I20260807 18:14:55.260887 27348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:55.261483 27204 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:55.261830 27204 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260807 18:14:55.261993 27204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-1/wal
uuid: "94d47e0385a840749cd3676abfda32c0"
format_stamp: "Formatted at 2026-08-07 18:14:54 on dist-test-slave-39l8"
server_key: "81c2f499b8e8c9afce97b5fa2b7ed3f7"
server_key_iv: "2e5f156eb83314856340d9a50261b826"
server_key_version: "encryptionkey@0"
I20260807 18:14:55.268251 27204 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:14:55.276090 27204 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 18:14:55.276340 27204 log.cc:1196] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 18:14:55.276858 27204 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0: Deleting consensus metadata
I20260807 18:14:55.277181 27204 master_runner.cc:315] Copying system tablet from 127.17.110.62:40597
I20260807 18:14:55.280407 27204 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0: tablet copy: Beginning tablet copy session from remote peer at address 127.17.110.62:40597
I20260807 18:14:55.305080 27361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:55.281675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.0.0.1:58348)
0807 18:14:55.288773 (+ 7098us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:55.288803 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:55.289020 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:55.289521 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:55.289526 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:55.290130 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:55.290474 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:55.290488 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.292118 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.292125 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:55.293663 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:55.293688 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.294904 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.294912 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:55.295109 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:55.295923 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:55.295955 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:55.297142 (+ 1187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:55.301588 (+ 4446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:55.301596 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:55.301601 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:55.302225 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:55.302771 (+ 546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:55.302777 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:55.302781 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:55.302971 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:55.303784 (+ 813us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:55.303790 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:55.303994 (+ 204us) client_negotiation.cc:769] Sending connection context
0807 18:14:55.304167 (+ 173us) client_negotiation.cc:240] Negotiation successful
0807 18:14:55.304569 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6899,"thread_start_us":6719,"threads_started":1}
I20260807 18:14:55.306285 27301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:55.281759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58348 (local address 127.17.110.62:40597)
0807 18:14:55.281979 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:55.281986 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:55.288978 (+ 6992us) server_negotiation.cc:407] Connection header received
0807 18:14:55.289214 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:55.289221 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:55.289318 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:55.289449 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:55.290725 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.291894 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.293898 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.294625 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.294833 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:55.297417 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:55.297424 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:55.297428 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:55.297474 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:55.301379 (+ 3905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:55.302401 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:55.302408 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:55.302412 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:55.302583 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:55.303139 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:55.303145 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:55.303148 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:55.303550 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:55.303703 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:55.305622 (+ 1919us) server_negotiation.cc:299] Negotiation successful
0807 18:14:55.305897 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 18:14:55.308115 26575 tablet_copy_service.cc:140] P ce9e1ae9d6354f9d9b7020b5e73f2ef2: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 94d47e0385a840749cd3676abfda32c0 ({username='oryx', principal='oryx/127.17.110.61@KRBTEST.COM'} at 127.0.0.1:58348)
I20260807 18:14:55.308658 26575 tablet_copy_service.cc:161] P ce9e1ae9d6354f9d9b7020b5e73f2ef2: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 94d47e0385a840749cd3676abfda32c0 at {username='oryx', principal='oryx/127.17.110.61@KRBTEST.COM'} at 127.0.0.1:58348: session id = 94d47e0385a840749cd3676abfda32c0-00000000000000000000000000000000
I20260807 18:14:55.313814 26575 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 18:14:55.317198 27204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 18:14:55.324975 27204 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0: tablet copy: Starting download of 0 data blocks...
I20260807 18:14:55.325462 27204 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0: tablet copy: Starting download of 1 WAL segments...
I20260807 18:14:55.329632 27204 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 18:14:55.335222 26575 tablet_copy_service.cc:342] P ce9e1ae9d6354f9d9b7020b5e73f2ef2: Request end of tablet copy session 94d47e0385a840749cd3676abfda32c0-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.17.110.61@KRBTEST.COM'} at 127.0.0.1:58348
I20260807 18:14:55.335511 26575 tablet_copy_service.cc:433] P ce9e1ae9d6354f9d9b7020b5e73f2ef2: ending tablet copy session 94d47e0385a840749cd3676abfda32c0-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 94d47e0385a840749cd3676abfda32c0
I20260807 18:14:55.352129 27204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:55.355266 27362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:55.356853 27364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:55.369117 27368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:55.372813 27204 server_base.cc:1034] running on GCE node
I20260807 18:14:55.373328 27204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:55.374208 27204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:55.375592 27204 hybrid_clock.cc:648] HybridClock initialized: now 1786126495375505 us; error 50 us; skew 500 ppm
I20260807 18:14:55.375857 27204 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:55 dist-test-slave-39l8 krb5kdc[26475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126495, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:55.378468 27204 init.cc:377] Logged in from keytab as oryx/127.17.110.61@KRBTEST.COM (short username oryx)
I20260807 18:14:55.380663 27204 webserver.cc:460] Webserver started at http://127.17.110.61:33095/ using document root <none> and password file <none>
I20260807 18:14:55.381187 27204 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:55.381338 27204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:55.385039 27204 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.002s
I20260807 18:14:55.387570 27376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:55.388104 27204 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 18:14:55.388321 27204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-1/wal
uuid: "94d47e0385a840749cd3676abfda32c0"
format_stamp: "Formatted at 2026-08-07 18:14:54 on dist-test-slave-39l8"
server_key: "81c2f499b8e8c9afce97b5fa2b7ed3f7"
server_key_iv: "2e5f156eb83314856340d9a50261b826"
server_key_version: "encryptionkey@0"
I20260807 18:14:55.388573 27204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:55.402390 27204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:55.407876 27204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:55.435613 27204 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.61:37317
I20260807 18:14:55.435675 27443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.61:37317 every 8 connection(s)
I20260807 18:14:55.437057 27204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-1/data/info.pb
I20260807 18:14:55.443662 27445 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:14:55.448230 27445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0: Bootstrap starting.
I20260807 18:14:55.470777 27445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:14:55.471465 27445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0: Bootstrap complete.
I20260807 18:14:55.472851 27445 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: NON_VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: true } }
I20260807 18:14:55.473196 27445 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 94d47e0385a840749cd3676abfda32c0, State: Initialized, Role: LEARNER
I20260807 18:14:55.473467 27445 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: NON_VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: true } }
I20260807 18:14:55.474457 27449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: NON_VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: true } } }
I20260807 18:14:55.474874 27445 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:14:55.475001 27449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [sys.catalog]: This master's current role is: LEARNER
I20260807 18:14:55.476850 27194 heartbeater.cc:503] Master 127.17.110.62:40597 was elected leader, sending a full tablet report...
I20260807 18:14:55.486732 27463 catalog_manager.cc:1259] Loaded cluster ID: 2749ae813a664a0f81bd97be8eb21721
I20260807 18:14:55.486876 27463 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0: loading cluster ID for follower catalog manager: success
I20260807 18:14:55.490913 27463 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0: acquiring CA information for follower catalog manager: success
I20260807 18:14:55.494217 27463 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 18:14:55.692945 27342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:55.693397 27342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:55.693591 27342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:55.701531 27342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:14:55.701728 27342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:55.701874 27342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:55.701977 27342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:14:55.702065 27342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:14:55.713017 27342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:43781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:40597,127.17.110.61:37317,127.17.110.60:34151
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.60.keytab
--principal=oryx/127.17.110.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.60:34151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.110.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:55.714687 27342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:55.716382 27342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:55.729789 27469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:55.729936 27473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:55.729941 27468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:55.735809 27342 server_base.cc:1034] running on GCE node
I20260807 18:14:55.736779 27342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:55.738515 27342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:55.739792 27342 hybrid_clock.cc:648] HybridClock initialized: now 1786126495739744 us; error 41 us; skew 500 ppm
I20260807 18:14:55.740226 27342 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:55 dist-test-slave-39l8 krb5kdc[26475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126495, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:55.744165 27342 init.cc:377] Logged in from keytab as oryx/127.17.110.60@KRBTEST.COM (short username oryx)
I20260807 18:14:55.746014 27342 webserver.cc:460] Webserver started at http://127.17.110.60:35077/ using document root <none> and password file <none>
I20260807 18:14:55.746927 27342 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:55.747097 27342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:55.747447 27342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:14:55.751920 27342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-2/data/instance:
uuid: "822d5b469a444f5ea371b43ecd44bb62"
format_stamp: "Formatted at 2026-08-07 18:14:55 on dist-test-slave-39l8"
server_key: "4fd4e871c1eb8c0c4823c0fde6ef2399"
server_key_iv: "88bca4ba9fc4178ff70265e120ceb57a"
server_key_version: "encryptionkey@0"
I20260807 18:14:55.752997 27342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-2/wal/instance:
uuid: "822d5b469a444f5ea371b43ecd44bb62"
format_stamp: "Formatted at 2026-08-07 18:14:55 on dist-test-slave-39l8"
server_key: "4fd4e871c1eb8c0c4823c0fde6ef2399"
server_key_iv: "88bca4ba9fc4178ff70265e120ceb57a"
server_key_version: "encryptionkey@0"
I20260807 18:14:55.758311 27483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:55.749895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:37317 (local address 127.0.0.1:47434)
0807 18:14:55.750478 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:55.750500 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:55.750660 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:55.751373 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:55.751379 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:55.751412 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:14:55.751632 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:55.751647 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.753446 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.753455 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:55.755657 (+ 2202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:55.755680 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.757352 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.757362 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:55.757567 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:55.757655 (+ 88us) client_negotiation.cc:769] Sending connection context
0807 18:14:55.757781 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 18:14:55.757941 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":334,"threads_started":1}
I20260807 18:14:55.758440 27484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:55.750139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47434 (local address 127.17.110.61:37317)
0807 18:14:55.750980 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:55.750985 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:55.751005 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:14:55.751075 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:55.751079 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:55.751160 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:55.751276 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:14:55.751875 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.753262 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.755905 (+ 2643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.757087 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.757246 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:55.757432 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:55.757825 (+ 393us) server_negotiation.cc:299] Negotiation successful
0807 18:14:55.757949 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":713,"thread_start_us":557,"threads_started":1}
I20260807 18:14:55.759708 27408 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260807 18:14:55.760396 27449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce9e1ae9d6354f9d9b7020b5e73f2ef2. Latest consensus state: current_term: 1 leader_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: NON_VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: true } } }
I20260807 18:14:55.760756 27449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [sys.catalog]: This master's current role is: LEARNER
I20260807 18:14:55.760836 27342 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.003s
I20260807 18:14:55.765437 27487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:55.767253 27342 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 18:14:55.767467 27342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-2/wal
uuid: "822d5b469a444f5ea371b43ecd44bb62"
format_stamp: "Formatted at 2026-08-07 18:14:55 on dist-test-slave-39l8"
server_key: "4fd4e871c1eb8c0c4823c0fde6ef2399"
server_key_iv: "88bca4ba9fc4178ff70265e120ceb57a"
server_key_version: "encryptionkey@0"
I20260807 18:14:55.767715 27342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:55.807945 27342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:55.813130 27342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:55.813552 27342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:55.837966 27342 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.60:34151
I20260807 18:14:55.838030 27560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.60:34151 every 8 connection(s)
I20260807 18:14:55.839699 27342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-2/data/info.pb
I20260807 18:14:55.843909 27561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:55.846885 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 27342
I20260807 18:14:55.851931 27561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:14:55 dist-test-slave-39l8 krb5kdc[26475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.110.60@KRBTEST.COM
Aug 07 18:14:55 dist-test-slave-39l8 krb5kdc[26475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126495, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.60@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM
I20260807 18:14:55.875049 27567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:55.856023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58356 (local address 127.17.110.62:40597)
0807 18:14:55.856628 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:55.856636 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:55.857407 (+ 771us) server_negotiation.cc:407] Connection header received
0807 18:14:55.857605 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:55.857609 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:55.857681 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:55.857782 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:55.859194 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.860062 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.862003 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.862916 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.863114 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:55.867733 (+ 4619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:55.867744 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:55.867750 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:55.867806 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:55.871448 (+ 3642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:55.872509 (+ 1061us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:55.872515 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:55.872519 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:55.872712 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:55.873213 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:55.873218 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:55.873221 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:55.873579 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:55.873756 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:55.874343 (+ 587us) server_negotiation.cc:299] Negotiation successful
0807 18:14:55.874599 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":291,"threads_started":1}
I20260807 18:14:55.876446 27569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:55.856014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.0.0.1:58356)
0807 18:14:55.857237 (+ 1223us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:55.857287 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:55.857476 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:55.857919 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:55.857930 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:55.858615 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:55.859004 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:55.859017 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.860270 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.860276 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:55.861803 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:55.861818 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.863228 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.863238 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:55.863385 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:55.864305 (+ 920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:55.864345 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:55.867444 (+ 3099us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:55.871603 (+ 4159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:55.871614 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:55.871629 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:55.872361 (+ 732us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:55.872899 (+ 538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:55.872902 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:55.872905 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:55.873070 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:55.873735 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:55.873744 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:55.873952 (+ 208us) client_negotiation.cc:769] Sending connection context
0807 18:14:55.874234 (+ 282us) client_negotiation.cc:240] Negotiation successful
0807 18:14:55.874926 (+ 692us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":880,"thread_start_us":566,"threads_started":1}
I20260807 18:14:55.876845 27566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:55.849486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46858 (local address 127.17.110.60:34151)
0807 18:14:55.850417 (+ 931us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:55.850430 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:55.850475 (+ 45us) server_negotiation.cc:407] Connection header received
0807 18:14:55.852366 (+ 1891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:55.852392 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:55.852929 (+ 537us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:55.853401 (+ 472us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:55.855539 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.857826 (+ 2287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.859467 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.860390 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.860616 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:55.866623 (+ 6007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:55.866636 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:55.866660 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:55.866746 (+ 86us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:55.870418 (+ 3672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:55.871603 (+ 1185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:55.871614 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:55.871625 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:55.871957 (+ 332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:55.872690 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:55.872696 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:55.872700 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:55.873615 (+ 915us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:55.875636 (+ 2021us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:55.875917 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 18:14:55.876258 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":698,"spinlock_wait_cycles":29952,"thread_start_us":465,"threads_started":1}
I20260807 18:14:55.878667 27561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:14:55 dist-test-slave-39l8 krb5kdc[26475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126495, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.60@KRBTEST.COM for oryx/127.17.110.61@KRBTEST.COM
I20260807 18:14:55.895099 27569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:55.879471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:37317 (local address 127.0.0.1:47444)
0807 18:14:55.879714 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:55.879735 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:55.879902 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:55.880328 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:55.880338 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:55.880757 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:55.880983 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:55.880994 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.882319 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.882324 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:55.883380 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:55.883394 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.884156 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.884161 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:55.884337 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:55.885075 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:55.885098 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:55.887748 (+ 2650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:55.891994 (+ 4246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:55.891999 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:55.892072 (+ 73us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:55.892613 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:55.893141 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:55.893146 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:55.893148 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:55.893365 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:55.893958 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:55.893963 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:55.894115 (+ 152us) client_negotiation.cc:769] Sending connection context
0807 18:14:55.894284 (+ 169us) client_negotiation.cc:240] Negotiation successful
0807 18:14:55.894566 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 18:14:55.895073 27484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:55.879750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47444 (local address 127.17.110.61:37317)
0807 18:14:55.879975 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:55.880006 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:55.880033 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:14:55.880103 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:55.880107 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:55.880173 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:55.880264 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:55.881364 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.882120 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.883542 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.883984 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.884094 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:55.888036 (+ 3942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:55.888042 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:55.888045 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:55.888087 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:55.891596 (+ 3509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:55.892785 (+ 1189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:55.892790 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:55.892793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:55.892951 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:55.893516 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:55.893520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:55.893523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:55.893836 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:55.893966 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:55.894352 (+ 386us) server_negotiation.cc:299] Negotiation successful
0807 18:14:55.894555 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 18:14:55.896433 27561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34151 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:14:55 dist-test-slave-39l8 krb5kdc[26475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126495, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.60@KRBTEST.COM for oryx/127.17.110.60@KRBTEST.COM
I20260807 18:14:55.912025 27566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:55.897480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46874 (local address 127.17.110.60:34151)
0807 18:14:55.897679 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:55.897683 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:55.897865 (+ 182us) server_negotiation.cc:407] Connection header received
0807 18:14:55.898021 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:55.898025 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:55.898089 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:55.898218 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:55.899041 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.899791 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.901085 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.901766 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.901894 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:55.905302 (+ 3408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:55.905308 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:55.905312 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:55.905347 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:55.908443 (+ 3096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:55.909412 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:55.909417 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:55.909419 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:55.909585 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:55.910192 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:55.910196 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:55.910198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:55.910515 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:55.910652 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:55.911271 (+ 619us) server_negotiation.cc:299] Negotiation successful
0807 18:14:55.911657 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":126,"server-negotiator.queue_time_us":74}
I20260807 18:14:55.912024 27569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:55.897555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:34151 (local address 127.0.0.1:46874)
0807 18:14:55.897767 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:55.897788 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:55.897917 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:55.898276 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:55.898280 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:55.898660 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:55.898897 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:55.898907 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.899957 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.899961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:55.900901 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:55.900915 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.901933 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.901937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:55.902121 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:55.902849 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:55.902869 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:55.905050 (+ 2181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:55.908652 (+ 3602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:55.908657 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:55.908661 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:55.909110 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:55.909753 (+ 643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:55.909757 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:55.909760 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:55.910005 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:55.910669 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:55.910673 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:55.910929 (+ 256us) client_negotiation.cc:769] Sending connection context
0807 18:14:55.911116 (+ 187us) client_negotiation.cc:240] Negotiation successful
0807 18:14:55.911475 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 18:14:55.918258 27561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62: Bootstrap starting.
I20260807 18:14:55.921958 27561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:55.923744 27561 log.cc:824] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62: Log is configured to *not* fsync() on all Append() calls
I20260807 18:14:55.927416 27561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62: No bootstrap required, opened a new log
I20260807 18:14:55.936204 27561 raft_consensus.cc:348] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34151 } }
I20260807 18:14:55.936616 27561 raft_consensus.cc:374] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:55.936707 27561 raft_consensus.cc:729] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 822d5b469a444f5ea371b43ecd44bb62, State: Initialized, Role: FOLLOWER
I20260807 18:14:55.937325 27561 consensus_queue.cc:260] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34151 } }
I20260807 18:14:55.938395 27573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34151 } } }
I20260807 18:14:55.938885 27573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:14:55.939630 27561 sys_catalog.cc:564] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 18:14:55.950445 27588 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:14:55.950644 27588 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:14:55.956256 27342 master_runner.cc:428] Detected that this master 822d5b469a444f5ea371b43ecd44bb62 is joining an existing cluster
I20260807 18:14:55.956451 27342 master_runner.cc:432] Initiating AddMaster RPC to add 127.17.110.60:34151
I20260807 18:14:55.983812 27567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:55.961934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58358 (local address 127.17.110.62:40597)
0807 18:14:55.962303 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:55.962311 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:55.963664 (+ 1353us) server_negotiation.cc:407] Connection header received
0807 18:14:55.964115 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:55.964121 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:55.964256 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:55.964358 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:55.966435 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.967675 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.969998 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.970757 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.970890 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:55.974453 (+ 3563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:55.974461 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:55.974466 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:55.974513 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:55.978906 (+ 4393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:55.979901 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:55.979907 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:55.979911 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:55.980103 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:55.981072 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:55.981078 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:55.981082 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:55.981507 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:55.981667 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:55.982398 (+ 731us) server_negotiation.cc:299] Negotiation successful
0807 18:14:55.982686 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186}
I20260807 18:14:55.985298 27595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:55.962504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.0.0.1:58358)
0807 18:14:55.963480 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:55.963511 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:55.963910 (+ 399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:55.964682 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:55.964690 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:55.965771 (+ 1081us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:55.966189 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:55.966216 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.967944 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.967950 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:55.969754 (+ 1804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:55.969776 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.971445 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.971459 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:55.971723 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:55.972692 (+ 969us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:55.972719 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:55.974141 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:55.979129 (+ 4988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:55.979136 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:55.979140 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:55.979735 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:55.980293 (+ 558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:55.980298 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:55.980301 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:55.980588 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:55.981785 (+ 1197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:55.981790 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:55.982030 (+ 240us) client_negotiation.cc:769] Sending connection context
0807 18:14:55.984372 (+ 2342us) client_negotiation.cc:240] Negotiation successful
0807 18:14:55.984775 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":756,"thread_start_us":569,"threads_started":1}
I20260807 18:14:55.985409 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 26639
I20260807 18:14:55.996065 27566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:55.961932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46888 (local address 127.17.110.60:34151)
0807 18:14:55.962243 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:55.962251 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:55.971275 (+ 9024us) server_negotiation.cc:407] Connection header received
0807 18:14:55.971574 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:55.971582 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:55.971675 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:55.971772 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:55.974152 (+ 2380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.975801 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.978745 (+ 2944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.979534 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.979631 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:55.982604 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:55.982614 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:55.982618 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:55.982661 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:55.992432 (+ 9771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:55.993101 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:55.993108 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:55.993112 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:55.993303 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:55.993777 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:55.993785 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:55.993791 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:55.994317 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:55.994515 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:55.995195 (+ 680us) server_negotiation.cc:299] Negotiation successful
0807 18:14:55.995598 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":115,"server-negotiator.queue_time_us":63}
I20260807 18:14:55.996567 27597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:55.962538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:34151 (local address 127.0.0.1:46888)
0807 18:14:55.971137 (+ 8599us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:55.971166 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:55.971415 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:55.972050 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:55.972057 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:55.973355 (+ 1298us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:55.973899 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:55.973919 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.976149 (+ 2230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.976158 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:55.978518 (+ 2360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:55.978542 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.980059 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.980079 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:55.980299 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:55.981051 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:55.981094 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:55.982381 (+ 1287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:55.992586 (+ 10205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:55.992590 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:55.992592 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:55.992961 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:55.993449 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:55.993453 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:55.993456 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:55.993616 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:55.994582 (+ 966us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:55.994586 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:55.994840 (+ 254us) client_negotiation.cc:769] Sending connection context
0807 18:14:55.995027 (+ 187us) client_negotiation.cc:240] Negotiation successful
0807 18:14:55.995401 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8368,"thread_start_us":7538,"threads_started":1}
I20260807 18:14:55.999221 27484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:55.965170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47448 (local address 127.17.110.61:37317)
0807 18:14:55.965427 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:55.965436 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:55.979007 (+ 13571us) server_negotiation.cc:407] Connection header received
0807 18:14:55.979332 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:55.979338 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:55.979449 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:55.979588 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:55.981409 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.982543 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.985036 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.985861 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.986000 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:55.989681 (+ 3681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:55.989691 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:55.989700 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:55.989765 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:55.994367 (+ 4602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:55.996084 (+ 1717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:55.996092 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:55.996096 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:55.996336 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:55.996960 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:55.996966 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:55.996973 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:55.997408 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:55.997581 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:55.998416 (+ 835us) server_negotiation.cc:299] Negotiation successful
0807 18:14:55.998780 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 18:14:56.000452 27599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:55.964784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:37317 (local address 127.0.0.1:47448)
0807 18:14:55.978883 (+ 14099us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:55.978908 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:55.979154 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:55.979797 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:55.979805 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:55.980742 (+ 937us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:55.981198 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:55.981215 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.983071 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.983077 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:55.984828 (+ 1751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:55.984848 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:55.986403 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:55.986410 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:55.986962 (+ 552us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:55.987770 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:55.987801 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:55.989351 (+ 1550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:55.994553 (+ 5202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:55.994562 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:55.994565 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:55.995158 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:55.996522 (+ 1364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:55.996530 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:55.996535 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:55.996797 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:55.997681 (+ 884us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:55.997686 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:55.998032 (+ 346us) client_negotiation.cc:769] Sending connection context
0807 18:14:55.999473 (+ 1441us) client_negotiation.cc:240] Negotiation successful
0807 18:14:55.999887 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13891,"thread_start_us":13685,"threads_started":1}
Aug 07 18:14:56 dist-test-slave-39l8 krb5kdc[26475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126492, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.62@KRBTEST.COM for oryx/127.17.110.60@KRBTEST.COM
Aug 07 18:14:56 dist-test-slave-39l8 krb5kdc[26475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.110.1@KRBTEST.COM
I20260807 18:14:56.016877 27483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:55.997283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:34151 (local address 127.0.0.1:46892)
0807 18:14:55.997668 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:55.997688 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:55.997804 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:55.998657 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:55.998662 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:55.999188 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:55.999406 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:55.999423 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:56.001027 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:56.001034 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:56.002480 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:56.002498 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:56.003680 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:56.003688 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:56.003936 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:56.004771 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:56.004802 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:56.007992 (+ 3190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:56.013290 (+ 5298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:56.013296 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:56.013300 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:56.013961 (+ 661us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:56.014633 (+ 672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:56.014637 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:56.014640 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:56.014881 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:56.015768 (+ 887us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:56.015773 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:56.015993 (+ 220us) client_negotiation.cc:769] Sending connection context
0807 18:14:56.016129 (+ 136us) client_negotiation.cc:240] Negotiation successful
0807 18:14:56.016470 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249}
I20260807 18:14:56.018627 27566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:55.998035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46892 (local address 127.17.110.60:34151)
0807 18:14:55.998218 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:55.998224 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:55.998243 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:14:55.998332 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:55.998337 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:55.998462 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:55.999575 (+ 1113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:55.999809 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:56.000856 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:56.002682 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:56.003462 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:56.003617 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:56.008443 (+ 4826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:56.008448 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:56.008452 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:56.008490 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:56.013023 (+ 4533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:56.014198 (+ 1175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:56.014206 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:56.014212 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:56.014385 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:56.015066 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:56.015074 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:56.015080 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:56.015505 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:56.017829 (+ 2324us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:56.017958 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 18:14:56.018250 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 18:14:56.019793 27600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:55.987408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56352 (local address 127.17.110.1:33421)
0807 18:14:55.993666 (+ 6258us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:55.993678 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:55.993724 (+ 46us) server_negotiation.cc:407] Connection header received
0807 18:14:55.993810 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:55.993816 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:55.994047 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:55.994267 (+ 220us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:55.996578 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:56.000262 (+ 3684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:56.002494 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:56.003581 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:56.004354 (+ 773us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:56.009759 (+ 5405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:56.009785 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:56.009813 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:56.009866 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:56.014361 (+ 4495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:56.015449 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:56.015462 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:56.015466 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:56.015655 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:56.016203 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:56.016208 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:56.016212 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:56.016673 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:56.018512 (+ 1839us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:56.019025 (+ 513us) server_negotiation.cc:299] Negotiation successful
0807 18:14:56.019368 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6038,"thread_start_us":5859,"threads_started":1}
I20260807 18:14:56.020411 26546 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.17.110.60:34151: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62"
member_type: NON_VOTER
last_known_addr {
host: "127.17.110.60"
port: 34151
}
attrs {
promote: true
}
}
dest_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2"
cas_config_opid_index: 5
I20260807 18:14:56.021632 26546 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: NON_VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: true } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: NON_VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: true } }
I20260807 18:14:56.023440 27480 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 822d5b469a444f5ea371b43ecd44bb62 (127.17.110.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: NON_VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: true } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: NON_VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: true } } }
I20260807 18:14:56.024286 27529 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:56.024474 26829 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:14:56.029784 27529 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [term 1 FOLLOWER]: Refusing update from remote peer ce9e1ae9d6354f9d9b7020b5e73f2ef2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 18:14:56.051558 27606 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: NON_VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 18:14:56.061919 27408 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [term 1 LEARNER]: Refusing update from remote peer ce9e1ae9d6354f9d9b7020b5e73f2ef2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 18:14:56.062809 27606 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: NON_VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260807 18:14:56.066802 27496 tablet.cc:2369] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:14:56.069439 27480 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.17.110.60:34151
I20260807 18:14:56.069815 27529 raft_consensus.cc:858] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34151 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: NON_VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: NON_VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: true } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: NON_VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: true } }
I20260807 18:14:56.070335 27342 master.cc:537] Master@127.17.110.60:34151 shutting down...
W20260807 18:14:56.070225 27529 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34151 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: NON_VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: NON_VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: true } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: NON_VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: true } }
I20260807 18:14:56.071986 27605 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: NON_VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: true } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: NON_VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: true } } }
I20260807 18:14:56.072436 27605 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:56.076831 27408 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 822d5b469a444f5ea371b43ecd44bb62 (127.17.110.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: NON_VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: true } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: NON_VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: true } } }
I20260807 18:14:56.082031 27449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: NON_VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: true } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: NON_VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: true } } }
I20260807 18:14:56.082371 27449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [sys.catalog]: This master's current role is: LEARNER
I20260807 18:14:56.083247 27606 raft_consensus.cc:1050] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2: attempting to promote NON_VOTER 94d47e0385a840749cd3676abfda32c0 to VOTER
I20260807 18:14:56.084182 27606 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: false } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: NON_VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: true } }
I20260807 18:14:56.087327 27408 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [term 1 LEARNER]: Refusing update from remote peer ce9e1ae9d6354f9d9b7020b5e73f2ef2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 18:14:56.088071 27605 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 18:14:56.091228 27606 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 94d47e0385a840749cd3676abfda32c0 (127.17.110.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: false } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: NON_VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: true } } }
I20260807 18:14:56.092067 27408 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 94d47e0385a840749cd3676abfda32c0 (127.17.110.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: false } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: NON_VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: true } } }
I20260807 18:14:56.101163 27449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: false } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: NON_VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: true } } }
I20260807 18:14:56.101497 27449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:14:56.104341 27605 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: false } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: NON_VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: true } } }
I20260807 18:14:56.104806 27605 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:56.105221 27606 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: false } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: NON_VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: true } } }
I20260807 18:14:56.105931 27606 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [sys.catalog]: This master's current role is: LEADER
W20260807 18:14:56.113620 26521 consensus_peers.cc:487] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 -> Peer 822d5b469a444f5ea371b43ecd44bb62 (127.17.110.60:34151): Couldn't send request to peer 822d5b469a444f5ea371b43ecd44bb62. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 18:14:56.128202 27529 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 94d47e0385a840749cd3676abfda32c0 (127.17.110.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: NON_VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: true } } }
I20260807 18:14:56.133903 27618 sys_catalog.cc:455] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: NON_VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: true } } }
I20260807 18:14:56.134483 27618 sys_catalog.cc:458] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260807 18:14:56.134439 27613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: NON_VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: true } } }
I20260807 18:14:56.134936 27613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260807 18:14:56.135013 27573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce9e1ae9d6354f9d9b7020b5e73f2ef2. Latest consensus state: current_term: 1 leader_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: NON_VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: true } } }
I20260807 18:14:56.135223 27573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260807 18:14:56.138577 27342 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260807 18:14:56.139165 27342 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260807 18:14:56.139380 27342 tablet_replica.cc:330] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62: stopping tablet replica
I20260807 18:14:56.148517 27342 master.cc:552] Master@127.17.110.60:34151 shutdown complete.
I20260807 18:14:56.148643 27342 master_runner.cc:305] Clearing existing system tablet
I20260807 18:14:56.149075 27342 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:56.149408 27342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:56.152882 27342 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:14:56.154878 27621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:56.155447 27342 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:56.156018 27342 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:14:56.156196 27342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-2/wal
uuid: "822d5b469a444f5ea371b43ecd44bb62"
format_stamp: "Formatted at 2026-08-07 18:14:55 on dist-test-slave-39l8"
server_key: "4fd4e871c1eb8c0c4823c0fde6ef2399"
server_key_iv: "88bca4ba9fc4178ff70265e120ceb57a"
server_key_version: "encryptionkey@0"
I20260807 18:14:56.161538 27342 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:14:56.169888 27342 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 18:14:56.170269 27342 log.cc:1196] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 18:14:56.171037 27342 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62: Deleting consensus metadata
I20260807 18:14:56.171398 27342 master_runner.cc:315] Copying system tablet from 127.17.110.62:40597
I20260807 18:14:56.174400 27342 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62: tablet copy: Beginning tablet copy session from remote peer at address 127.17.110.62:40597
I20260807 18:14:56.192476 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 26639
I20260807 18:14:56.196770 27630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:56.175433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.0.0.1:58362)
0807 18:14:56.176167 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:56.176197 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:56.176469 (+ 272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:56.177088 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:56.177094 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:56.177712 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:56.177992 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:56.178006 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:56.179685 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:56.179700 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:56.181407 (+ 1707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:56.181430 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:56.183061 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:56.183068 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:56.183290 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:56.184219 (+ 929us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:56.184245 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:56.185437 (+ 1192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:56.190729 (+ 5292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:56.190738 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:56.190743 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:56.191357 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:56.191875 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:56.191881 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:56.191885 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:56.192120 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:56.195407 (+ 3287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:56.195414 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:56.195678 (+ 264us) client_negotiation.cc:769] Sending connection context
0807 18:14:56.195850 (+ 172us) client_negotiation.cc:240] Negotiation successful
0807 18:14:56.196265 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":374,"threads_started":1}
I20260807 18:14:56.197860 27567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:56.175491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58362 (local address 127.17.110.62:40597)
0807 18:14:56.175776 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:56.175786 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:56.176327 (+ 541us) server_negotiation.cc:407] Connection header received
0807 18:14:56.176667 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:56.176674 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:56.176765 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:56.176853 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:56.178199 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:56.179461 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:56.182033 (+ 2572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:56.182842 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:56.182991 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:56.185701 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:56.185708 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:56.185714 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:56.185761 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:56.190483 (+ 4722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:56.191520 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:56.191526 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:56.191529 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:56.191719 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:56.192282 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:56.192288 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:56.192294 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:56.195123 (+ 2829us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:56.195308 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:56.197181 (+ 1873us) server_negotiation.cc:299] Negotiation successful
0807 18:14:56.197464 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260807 18:14:56.198374 26575 tablet_copy_service.cc:140] P ce9e1ae9d6354f9d9b7020b5e73f2ef2: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 822d5b469a444f5ea371b43ecd44bb62 ({username='oryx', principal='oryx/127.17.110.60@KRBTEST.COM'} at 127.0.0.1:58362)
I20260807 18:14:56.198607 26575 tablet_copy_service.cc:161] P ce9e1ae9d6354f9d9b7020b5e73f2ef2: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 822d5b469a444f5ea371b43ecd44bb62 at {username='oryx', principal='oryx/127.17.110.60@KRBTEST.COM'} at 127.0.0.1:58362: session id = 822d5b469a444f5ea371b43ecd44bb62-00000000000000000000000000000000
I20260807 18:14:56.202438 26575 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 18:14:56.206310 27342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 18:14:56.214224 27342 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62: tablet copy: Starting download of 0 data blocks...
I20260807 18:14:56.214614 27342 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62: tablet copy: Starting download of 1 WAL segments...
I20260807 18:14:56.217782 27342 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 18:14:56.223205 26575 tablet_copy_service.cc:342] P ce9e1ae9d6354f9d9b7020b5e73f2ef2: Request end of tablet copy session 822d5b469a444f5ea371b43ecd44bb62-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.17.110.60@KRBTEST.COM'} at 127.0.0.1:58362
I20260807 18:14:56.223414 26575 tablet_copy_service.cc:433] P ce9e1ae9d6354f9d9b7020b5e73f2ef2: ending tablet copy session 822d5b469a444f5ea371b43ecd44bb62-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 822d5b469a444f5ea371b43ecd44bb62
I20260807 18:14:56.228986 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.1:33421
--local_ip_for_outbound_sockets=127.17.110.1
--tserver_master_addrs=127.17.110.62:40597
--webserver_port=40233
--webserver_interface=127.17.110.1
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.1.keytab
--principal=oryx/127.17.110.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:43781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.110.62:40597,127.17.110.61:37317
--tserver_master_addrs=127.17.110.62:40597,127.17.110.61:37317,127.17.110.60:34151 with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:14:56.238272 27342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:56.242271 27632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:56.247789 27634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:56.256871 27342 server_base.cc:1034] running on GCE node
W20260807 18:14:56.257764 27636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:56.258385 27342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:56.259275 27342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:56.260507 27342 hybrid_clock.cc:648] HybridClock initialized: now 1786126496260484 us; error 91 us; skew 500 ppm
I20260807 18:14:56.260967 27342 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:56 dist-test-slave-39l8 krb5kdc[26475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126496, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:56.265043 27342 init.cc:377] Logged in from keytab as oryx/127.17.110.60@KRBTEST.COM (short username oryx)
I20260807 18:14:56.267938 27342 webserver.cc:460] Webserver started at http://127.17.110.60:37141/ using document root <none> and password file <none>
I20260807 18:14:56.268373 27342 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:56.268494 27342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:56.272307 27342 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:14:56.274756 27646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:56.275367 27342 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 18:14:56.275516 27342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-2/wal
uuid: "822d5b469a444f5ea371b43ecd44bb62"
format_stamp: "Formatted at 2026-08-07 18:14:55 on dist-test-slave-39l8"
server_key: "4fd4e871c1eb8c0c4823c0fde6ef2399"
server_key_iv: "88bca4ba9fc4178ff70265e120ceb57a"
server_key_version: "encryptionkey@0"
I20260807 18:14:56.275767 27342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:56.303977 27342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:56.309242 27342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:56.338904 27342 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.60:34151
I20260807 18:14:56.338965 27722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.60:34151 every 8 connection(s)
I20260807 18:14:56.340183 27342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/master-2/data/info.pb
I20260807 18:14:56.348308 27723 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:14:56.352468 27723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62: Bootstrap starting.
I20260807 18:14:56.375260 27723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:14:56.376022 27723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62: Bootstrap complete.
I20260807 18:14:56.377410 27723 raft_consensus.cc:348] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: false } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: NON_VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: true } }
I20260807 18:14:56.377916 27723 raft_consensus.cc:729] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 822d5b469a444f5ea371b43ecd44bb62, State: Initialized, Role: LEARNER
I20260807 18:14:56.378288 27723 consensus_queue.cc:260] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: false } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: NON_VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: true } }
I20260807 18:14:56.385397 27726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: false } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: NON_VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: true } } }
I20260807 18:14:56.385878 27726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [sys.catalog]: This master's current role is: LEARNER
I20260807 18:14:56.386690 27723 sys_catalog.cc:564] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:14:56.411612 27742 catalog_manager.cc:1259] Loaded cluster ID: 2749ae813a664a0f81bd97be8eb21721
I20260807 18:14:56.411753 27742 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62: loading cluster ID for follower catalog manager: success
I20260807 18:14:56.416128 27742 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62: acquiring CA information for follower catalog manager: success
I20260807 18:14:56.418895 27742 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:14:56.491425 27483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:56.480875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:34151 (local address 127.0.0.1:46904)
0807 18:14:56.481349 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:56.481376 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:56.481561 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:56.483756 (+ 2195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:56.483762 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:56.483806 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:14:56.484116 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:56.484137 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:56.486378 (+ 2241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:56.486387 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:56.488401 (+ 2014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:56.488427 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:56.490451 (+ 2024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:56.490461 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:56.490729 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:56.490772 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 18:14:56.490888 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 18:14:56.491078 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309}
I20260807 18:14:56.491926 27743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:56.481987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46904 (local address 127.17.110.60:34151)
0807 18:14:56.483265 (+ 1278us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:56.483272 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:56.483295 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:14:56.483380 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:56.483386 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:56.483488 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:56.483666 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:14:56.484330 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:56.486164 (+ 1834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:56.488624 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:56.490156 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:56.490354 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:56.490675 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:56.490951 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 18:14:56.491126 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1109,"thread_start_us":927,"threads_started":1}
I20260807 18:14:56.493091 27686 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260807 18:14:56.493585 27726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce9e1ae9d6354f9d9b7020b5e73f2ef2. Latest consensus state: current_term: 1 leader_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: false } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: NON_VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: true } } }
I20260807 18:14:56.493908 27726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [sys.catalog]: This master's current role is: LEARNER
W20260807 18:14:56.685353 27631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:56.685679 27631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:56.685842 27631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:56.694339 27631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:56.694480 27631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:56.694670 27631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.1
I20260807 18:14:56.705368 27631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:43781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.1.keytab
--principal=oryx/127.17.110.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.1:33421
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.110.1
--webserver_port=40233
--tserver_master_addrs=127.17.110.62:40597,127.17.110.61:37317,127.17.110.60:34151
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.1
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:56.707012 27631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:56.708479 27631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:56.723699 27751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:56.723759 27756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:56.723783 27752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:56.724874 27631 server_base.cc:1034] running on GCE node
I20260807 18:14:56.725766 27631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:56.726951 27631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:56.728216 27631 hybrid_clock.cc:648] HybridClock initialized: now 1786126496728174 us; error 38 us; skew 500 ppm
I20260807 18:14:56.728600 27631 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:56 dist-test-slave-39l8 krb5kdc[26475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126496, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:56.732424 27631 init.cc:377] Logged in from keytab as oryx/127.17.110.1@KRBTEST.COM (short username oryx)
I20260807 18:14:56.734189 27631 webserver.cc:460] Webserver started at http://127.17.110.1:40233/ using document root <none> and password file <none>
I20260807 18:14:56.735033 27631 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:56.735162 27631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:56.741089 27631 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260807 18:14:56.744316 27766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:56.745899 27631 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:14:56.746106 27631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-0/wal
uuid: "d6fda665629f44b69b25453b35fb9490"
format_stamp: "Formatted at 2026-08-07 18:14:53 on dist-test-slave-39l8"
server_key: "875295331fd56895dfe5cdd9963fe0ab"
server_key_iv: "4419ff81956ad9987e3aa58d8673a6f6"
server_key_version: "encryptionkey@0"
I20260807 18:14:56.747248 27631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:56.792411 27631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:56.797760 27631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:56.798122 27631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:56.799299 27631 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:14:56.800912 27631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:14:56.801011 27631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:56.801139 27631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:14:56.801249 27631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:56.841566 27631 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.1:33421
I20260807 18:14:56.841602 27929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.1:33421 every 8 connection(s)
I20260807 18:14:56.843287 27631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
I20260807 18:14:56.844005 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 27631
I20260807 18:14:56.844484 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 26843
Aug 07 18:14:56 dist-test-slave-39l8 krb5kdc[26475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.110.2@KRBTEST.COM
Aug 07 18:14:56 dist-test-slave-39l8 krb5kdc[26475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126496, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.61@KRBTEST.COM
Aug 07 18:14:56 dist-test-slave-39l8 krb5kdc[26475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126496, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM
I20260807 18:14:56.883198 27936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:56.845601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58622 (local address 127.17.110.2:40847)
0807 18:14:56.849799 (+ 4198us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:56.849807 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:56.849836 (+ 29us) server_negotiation.cc:407] Connection header received
0807 18:14:56.849902 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:56.849909 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:56.850115 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:56.850325 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:56.853643 (+ 3318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:56.855983 (+ 2340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:56.859029 (+ 3046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:56.860320 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:56.860516 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:56.867426 (+ 6910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:56.867451 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:56.867471 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:56.867528 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:56.871818 (+ 4290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:56.872987 (+ 1169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:56.873000 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:56.873004 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:56.878947 (+ 5943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:56.879616 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:56.879623 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:56.879626 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:56.880327 (+ 701us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:56.880680 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:56.882338 (+ 1658us) server_negotiation.cc:299] Negotiation successful
0807 18:14:56.882690 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4015,"thread_start_us":3819,"threads_started":1}
I20260807 18:14:56.884734 27005 generic_service.cc:196] Checking for leaks (request via RPC)
Aug 07 18:14:56 dist-test-slave-39l8 krb5kdc[26475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126496, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.60@KRBTEST.COM
I20260807 18:14:56.895162 27942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:56.866042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:36803 (local address 127.17.110.61:37317)
0807 18:14:56.866671 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:56.866678 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:56.867484 (+ 806us) server_negotiation.cc:407] Connection header received
0807 18:14:56.869363 (+ 1879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:56.869371 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:56.869491 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:56.869580 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:56.873638 (+ 4058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:56.874400 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:56.876372 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:56.877186 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:56.877373 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:56.885093 (+ 7720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:56.885102 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:56.885120 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:56.885168 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:56.889722 (+ 4554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:56.891159 (+ 1437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:56.891170 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:56.891176 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:56.891389 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:56.892428 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:56.892435 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:56.892439 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:56.892914 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:56.893102 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:56.894340 (+ 1238us) server_negotiation.cc:299] Negotiation successful
0807 18:14:56.894564 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":354,"threads_started":1}
I20260807 18:14:56.896087 27941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:56.848913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:42425 (local address 127.17.110.62:40597)
0807 18:14:56.863604 (+ 14691us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:56.863613 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:56.866173 (+ 2560us) server_negotiation.cc:407] Connection header received
0807 18:14:56.868834 (+ 2661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:56.868842 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:56.868950 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:56.869072 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:56.872404 (+ 3332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:56.873351 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:56.876518 (+ 3167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:56.877237 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:56.877372 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:56.885134 (+ 7762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:56.885141 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:56.885145 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:56.885190 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:56.891611 (+ 6421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:56.892792 (+ 1181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:56.892799 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:56.892803 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:56.893002 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:56.893570 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:56.893579 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:56.893586 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:56.894015 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:56.894215 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:56.894856 (+ 641us) server_negotiation.cc:299] Negotiation successful
0807 18:14:56.895156 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14489,"thread_start_us":14320,"threads_started":1}
I20260807 18:14:56.897360 27943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:56.866669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:37317 (local address 127.17.110.1:36803)
0807 18:14:56.867342 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:56.867366 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:56.868542 (+ 1176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:56.869912 (+ 1370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:56.869917 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:56.870780 (+ 863us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:56.872127 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:56.872163 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:56.874568 (+ 2405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:56.874575 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:56.876154 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:56.876176 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:56.877717 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:56.877723 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:56.877974 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:56.879217 (+ 1243us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:56.879255 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:56.884768 (+ 5513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:56.889929 (+ 5161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:56.889942 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:56.889967 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:56.890893 (+ 926us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:56.891624 (+ 731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:56.891630 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:56.891633 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:56.892271 (+ 638us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:56.893202 (+ 931us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:56.893211 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:56.893776 (+ 565us) client_negotiation.cc:769] Sending connection context
0807 18:14:56.895435 (+ 1659us) client_negotiation.cc:240] Negotiation successful
0807 18:14:56.895804 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"mutex_wait_us":164,"spinlock_wait_cycles":43392,"thread_start_us":319,"threads_started":1}
I20260807 18:14:56.897392 27938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:56.848108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.17.110.1:42425)
0807 18:14:56.865705 (+ 17597us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:56.865726 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:56.868397 (+ 2671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:56.869456 (+ 1059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:56.869471 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:56.870781 (+ 1310us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:56.872127 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:56.872162 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:56.873774 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:56.873781 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:56.876334 (+ 2553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:56.876354 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:56.877535 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:56.877545 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:56.877981 (+ 436us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:56.879196 (+ 1215us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:56.879237 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:56.884264 (+ 5027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:56.891734 (+ 7470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:56.891739 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:56.891744 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:56.892576 (+ 832us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:56.893141 (+ 565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:56.893150 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:56.893156 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:56.893396 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:56.894164 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:56.894172 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:56.894405 (+ 233us) client_negotiation.cc:769] Sending connection context
0807 18:14:56.894632 (+ 227us) client_negotiation.cc:240] Negotiation successful
0807 18:14:56.895497 (+ 865us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17441,"mutex_wait_us":140,"spinlock_wait_cycles":124288,"thread_start_us":15747,"threads_started":1}
I20260807 18:14:56.899773 27932 heartbeater.cc:347] Connected to a master server at 127.17.110.61:37317
I20260807 18:14:56.900153 27932 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:56.901254 27932 heartbeater.cc:511] Master 127.17.110.61:37317 requested a full tablet report, sending...
I20260807 18:14:56.902166 27931 heartbeater.cc:347] Connected to a master server at 127.17.110.62:40597
I20260807 18:14:56.902334 27931 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:56.902908 27931 heartbeater.cc:511] Master 127.17.110.62:40597 requested a full tablet report, sending...
I20260807 18:14:56.904276 27394 ts_manager.cc:194] Registered new tserver with Master: d6fda665629f44b69b25453b35fb9490 (127.17.110.1:33421)
I20260807 18:14:56.904807 26546 ts_manager.cc:194] Re-registered known tserver with Master: d6fda665629f44b69b25453b35fb9490 (127.17.110.1:33421)
I20260807 18:14:56.906050 26546 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.110.1@KRBTEST.COM'} at 127.17.110.1:42425
I20260807 18:14:56.907855 27743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:56.847614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:60421 (local address 127.17.110.60:34151)
0807 18:14:56.848523 (+ 909us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:56.848531 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:56.865152 (+ 16621us) server_negotiation.cc:407] Connection header received
0807 18:14:56.868896 (+ 3744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:56.868903 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:56.868999 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:56.869110 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:56.872414 (+ 3304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:56.873541 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:56.880648 (+ 7107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:56.881572 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:56.881689 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:56.898251 (+ 16562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:56.898272 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:56.898281 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:56.898354 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:56.903320 (+ 4966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:56.905148 (+ 1828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:56.905161 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:56.905163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:56.905329 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:56.905817 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:56.905823 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:56.905827 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:56.906267 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:56.906469 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:56.907174 (+ 705us) server_negotiation.cc:299] Negotiation successful
0807 18:14:56.907499 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":742}
I20260807 18:14:56.909054 27939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:56.847877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:34151 (local address 127.17.110.1:60421)
0807 18:14:56.864934 (+ 17057us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:56.865010 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:56.868397 (+ 3387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:56.869456 (+ 1059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:56.869471 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:56.870781 (+ 1310us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:56.872127 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:56.872162 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:56.878136 (+ 5974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:56.878142 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:56.880344 (+ 2202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:56.880371 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:56.882026 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:56.882035 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:56.882356 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:56.883513 (+ 1157us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:56.883555 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:56.893862 (+ 10307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:56.903608 (+ 9746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:56.903615 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:56.903622 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:56.904386 (+ 764us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:56.905472 (+ 1086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:56.905475 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:56.905478 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:56.905627 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:56.906438 (+ 811us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:56.906443 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:56.906797 (+ 354us) client_negotiation.cc:769] Sending connection context
0807 18:14:56.908155 (+ 1358us) client_negotiation.cc:240] Negotiation successful
0807 18:14:56.908592 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16112,"mutex_wait_us":251,"spinlock_wait_cycles":47360,"thread_start_us":17268,"threads_started":1}
I20260807 18:14:56.910166 27930 heartbeater.cc:347] Connected to a master server at 127.17.110.60:34151
I20260807 18:14:56.910354 27930 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:56.910916 27930 heartbeater.cc:511] Master 127.17.110.60:34151 requested a full tablet report, sending...
I20260807 18:14:56.913245 27673 ts_manager.cc:194] Registered new tserver with Master: d6fda665629f44b69b25453b35fb9490 (127.17.110.1:33421)
I20260807 18:14:56.996446 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 26843
W20260807 18:14:57.023511 26522 connection.cc:535] server connection from 127.17.110.2:47623 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 18:14:57.024398 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.2:40847
--local_ip_for_outbound_sockets=127.17.110.2
--tserver_master_addrs=127.17.110.62:40597
--webserver_port=33255
--webserver_interface=127.17.110.2
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.2.keytab
--principal=oryx/127.17.110.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:43781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.110.62:40597,127.17.110.61:37317
--tserver_master_addrs=127.17.110.62:40597,127.17.110.61:37317,127.17.110.60:34151 with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:14:57.096630 27606 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: false } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: NON_VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: true } } }
I20260807 18:14:57.097039 27606 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:57.098336 27606 raft_consensus.cc:1050] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2: attempting to promote NON_VOTER 822d5b469a444f5ea371b43ecd44bb62 to VOTER
I20260807 18:14:57.099283 27606 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: false } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: false } }
I20260807 18:14:57.101784 27686 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [term 1 LEARNER]: Refusing update from remote peer ce9e1ae9d6354f9d9b7020b5e73f2ef2: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260807 18:14:57.101799 27408 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [term 1 FOLLOWER]: Refusing update from remote peer ce9e1ae9d6354f9d9b7020b5e73f2ef2: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260807 18:14:57.102581 27950 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 18:14:57.103128 27949 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 18:14:57.107239 27950 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 822d5b469a444f5ea371b43ecd44bb62 (127.17.110.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: false } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: false } } }
I20260807 18:14:57.111781 27408 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 822d5b469a444f5ea371b43ecd44bb62 (127.17.110.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: false } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: false } } }
I20260807 18:14:57.113250 27950 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: false } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: false } } }
I20260807 18:14:57.113523 27949 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: false } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: false } } }
I20260807 18:14:57.113737 27950 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:57.113852 27949 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:57.114041 27950 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: false } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: false } } }
I20260807 18:14:57.114333 27950 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2 [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:57.114580 27686 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 822d5b469a444f5ea371b43ecd44bb62 (127.17.110.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: false } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: false } } }
I20260807 18:14:57.116670 27956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: false } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: false } } }
I20260807 18:14:57.117316 27956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:14:57.119679 27957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: false } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: false } } }
I20260807 18:14:57.120177 27957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 18:14:57.425617 27947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:57.426060 27947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:57.426177 27947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:57.434168 27947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:57.434329 27947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:57.434448 27947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.2
I20260807 18:14:57.445207 27947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:43781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.2.keytab
--principal=oryx/127.17.110.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.2:40847
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.110.2
--webserver_port=33255
--tserver_master_addrs=127.17.110.62:40597,127.17.110.61:37317,127.17.110.60:34151
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.2
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:57.446779 27947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:57.448289 27947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:57.465071 27962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:57.465214 27966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:57.465081 27963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:57.466151 27947 server_base.cc:1034] running on GCE node
I20260807 18:14:57.467072 27947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:57.468307 27947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:57.469580 27947 hybrid_clock.cc:648] HybridClock initialized: now 1786126497469519 us; error 106 us; skew 500 ppm
I20260807 18:14:57.470116 27947 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:57 dist-test-slave-39l8 krb5kdc[26475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126497, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:57.474432 27947 init.cc:377] Logged in from keytab as oryx/127.17.110.2@KRBTEST.COM (short username oryx)
I20260807 18:14:57.476806 27947 webserver.cc:460] Webserver started at http://127.17.110.2:33255/ using document root <none> and password file <none>
I20260807 18:14:57.477798 27947 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:57.477995 27947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:57.486259 27947 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:14:57.490398 27976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:57.492553 27947 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 18:14:57.492857 27947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-1/wal
uuid: "480a54d1b82d42acb4428942491d0910"
format_stamp: "Formatted at 2026-08-07 18:14:53 on dist-test-slave-39l8"
server_key: "5678041c1c6867cf3e718ce526290c6c"
server_key_iv: "109cf82c5eae88b673e79e24da40be1a"
server_key_version: "encryptionkey@0"
I20260807 18:14:57.494107 27947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:57.530038 27947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:57.540560 27947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:57.540975 27947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:57.542464 27947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:14:57.544754 27947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:14:57.544938 27947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:57.545081 27947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:14:57.545179 27947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:57.592823 27947 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.2:40847
I20260807 18:14:57.592895 28123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.2:40847 every 8 connection(s)
I20260807 18:14:57.594616 27947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
I20260807 18:14:57.600395 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 27947
I20260807 18:14:57.600798 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 27021
Aug 07 18:14:57 dist-test-slave-39l8 krb5kdc[26475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.110.3@KRBTEST.COM
Aug 07 18:14:57 dist-test-slave-39l8 krb5kdc[26475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126497, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM
Aug 07 18:14:57 dist-test-slave-39l8 krb5kdc[26475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126497, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.60@KRBTEST.COM
Aug 07 18:14:57 dist-test-slave-39l8 krb5kdc[26475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126497, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.61@KRBTEST.COM
I20260807 18:14:57.659976 28136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:57.602599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50248 (local address 127.17.110.3:37809)
0807 18:14:57.631552 (+ 28953us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:57.631566 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:57.632396 (+ 830us) server_negotiation.cc:407] Connection header received
0807 18:14:57.632657 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:57.632668 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:57.632956 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:57.633140 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:57.635760 (+ 2620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:57.637430 (+ 1670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:57.639510 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:57.640552 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:57.640697 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:57.649914 (+ 9217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:57.649941 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:57.649968 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:57.650038 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:57.655694 (+ 5656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:57.656956 (+ 1262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:57.656971 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:57.656976 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:57.657195 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:57.657737 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:57.657752 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:57.657756 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:57.658285 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:57.658653 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:57.659091 (+ 438us) server_negotiation.cc:299] Negotiation successful
0807 18:14:57.659430 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28718,"thread_start_us":28514,"threads_started":1}
I20260807 18:14:57.660967 28129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:57.598341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:42233 (local address 127.17.110.62:40597)
0807 18:14:57.628828 (+ 30487us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:57.628838 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:57.637658 (+ 8820us) server_negotiation.cc:407] Connection header received
0807 18:14:57.637737 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:57.637744 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:57.637843 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:57.637973 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:57.639929 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:57.641043 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:57.643991 (+ 2948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:57.644685 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:57.644833 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:57.650418 (+ 5585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:57.650424 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:57.650427 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:57.650465 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:57.654744 (+ 4279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:57.657214 (+ 2470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:57.657219 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:57.657223 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:57.657448 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:57.658164 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:57.658169 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:57.658172 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:57.658520 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:57.658690 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:57.660249 (+ 1559us) server_negotiation.cc:299] Negotiation successful
0807 18:14:57.660539 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30324,"thread_start_us":32448,"threads_started":1}
I20260807 18:14:57.662178 28130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:57.598422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:41921 (local address 127.17.110.60:34151)
0807 18:14:57.629315 (+ 30893us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:57.629324 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:57.629354 (+ 30us) server_negotiation.cc:407] Connection header received
0807 18:14:57.633245 (+ 3891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:57.633253 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:57.633379 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:57.633518 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:57.637097 (+ 3579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:57.639163 (+ 2066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:57.645676 (+ 6513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:57.646410 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:57.646548 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:57.652480 (+ 5932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:57.652490 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:57.652495 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:57.652547 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:57.657944 (+ 5397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:57.659007 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:57.659013 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:57.659017 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:57.659201 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:57.659742 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:57.659747 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:57.659751 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:57.660500 (+ 749us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:57.660707 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:57.661438 (+ 731us) server_negotiation.cc:299] Negotiation successful
0807 18:14:57.661707 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30704,"thread_start_us":30590,"threads_started":1}
I20260807 18:14:57.663091 28137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:57.631749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:36101 (local address 127.17.110.61:37317)
0807 18:14:57.638517 (+ 6768us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:57.638524 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:57.640568 (+ 2044us) server_negotiation.cc:407] Connection header received
0807 18:14:57.640822 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:57.640830 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:57.640927 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:57.641052 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:57.642539 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:57.643322 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:57.645318 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:57.646026 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:57.646150 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:57.651756 (+ 5606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:57.651765 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:57.651771 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:57.651824 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:57.655851 (+ 4027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:57.657703 (+ 1852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:57.657709 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:57.657712 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:57.657908 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:57.659117 (+ 1209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:57.659123 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:57.659127 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:57.659476 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:57.659646 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:57.662106 (+ 2460us) server_negotiation.cc:299] Negotiation successful
0807 18:14:57.662689 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6616,"thread_start_us":6502,"threads_started":1}
I20260807 18:14:57.662385 28131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:57.599329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:34151 (local address 127.17.110.2:41921)
0807 18:14:57.627898 (+ 28569us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:57.627967 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:57.633013 (+ 5046us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:57.634203 (+ 1190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:57.634222 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:57.635477 (+ 1255us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:57.636721 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:57.636773 (+ 52us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:57.639426 (+ 2653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:57.639432 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:57.645470 (+ 6038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:57.645482 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:57.646846 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:57.646853 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:57.647074 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:57.647681 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:57.647702 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:57.652139 (+ 4437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:57.658140 (+ 6001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:57.658145 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:57.658149 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:57.658787 (+ 638us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:57.659385 (+ 598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:57.659391 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:57.659395 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:57.659587 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:57.660649 (+ 1062us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:57.660657 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:57.660865 (+ 208us) client_negotiation.cc:769] Sending connection context
0807 18:14:57.661039 (+ 174us) client_negotiation.cc:240] Negotiation successful
0807 18:14:57.661398 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28143,"mutex_wait_us":1138,"thread_start_us":27778,"threads_started":1}
I20260807 18:14:57.662441 28138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:57.637793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:37317 (local address 127.17.110.2:36101)
0807 18:14:57.640430 (+ 2637us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:57.640449 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:57.640620 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:57.641129 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:57.641137 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:57.641905 (+ 768us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:57.642317 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:57.642331 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:57.643476 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:57.643484 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:57.645130 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:57.645145 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:57.646434 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:57.646440 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:57.646665 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:57.647403 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:57.647435 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:57.651371 (+ 3936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:57.656025 (+ 4654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:57.656037 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:57.656059 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:57.657062 (+ 1003us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:57.658053 (+ 991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:57.658112 (+ 59us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:57.658118 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:57.658958 (+ 840us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:57.659711 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:57.659716 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:57.659851 (+ 135us) client_negotiation.cc:769] Sending connection context
0807 18:14:57.661244 (+ 1393us) client_negotiation.cc:240] Negotiation successful
0807 18:14:57.661555 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2486,"mutex_wait_us":27,"thread_start_us":2286,"threads_started":1}
I20260807 18:14:57.663925 27189 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:14:57.662685 28133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:57.599329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.17.110.2:42233)
0807 18:14:57.631330 (+ 32001us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:57.631355 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:57.632447 (+ 1092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:57.638021 (+ 5574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:57.638050 (+ 29us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:57.639081 (+ 1031us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:57.639705 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:57.639728 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:57.641192 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:57.641204 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:57.643784 (+ 2580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:57.643805 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:57.645020 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:57.645028 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:57.645410 (+ 382us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:57.646391 (+ 981us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:57.646433 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:57.650106 (+ 3673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:57.656050 (+ 5944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:57.656053 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:57.656071 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:57.657054 (+ 983us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:57.657726 (+ 672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:57.657730 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:57.657733 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:57.657998 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:57.659261 (+ 1263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:57.659270 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:57.659754 (+ 484us) client_negotiation.cc:769] Sending connection context
0807 18:14:57.660128 (+ 374us) client_negotiation.cc:240] Negotiation successful
0807 18:14:57.660698 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31791,"mutex_wait_us":583,"spinlock_wait_cycles":41088,"thread_start_us":31010,"threads_started":1}
I20260807 18:14:57.665401 28126 heartbeater.cc:347] Connected to a master server at 127.17.110.61:37317
I20260807 18:14:57.665442 28124 heartbeater.cc:347] Connected to a master server at 127.17.110.60:34151
I20260807 18:14:57.665820 28126 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:57.665828 28124 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:57.666771 28125 heartbeater.cc:347] Connected to a master server at 127.17.110.62:40597
I20260807 18:14:57.666962 28125 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:57.667165 28126 heartbeater.cc:511] Master 127.17.110.61:37317 requested a full tablet report, sending...
I20260807 18:14:57.667260 28124 heartbeater.cc:511] Master 127.17.110.60:34151 requested a full tablet report, sending...
I20260807 18:14:57.667845 28125 heartbeater.cc:511] Master 127.17.110.62:40597 requested a full tablet report, sending...
I20260807 18:14:57.669338 27394 ts_manager.cc:194] Registered new tserver with Master: 480a54d1b82d42acb4428942491d0910 (127.17.110.2:40847)
I20260807 18:14:57.669551 27673 ts_manager.cc:194] Registered new tserver with Master: 480a54d1b82d42acb4428942491d0910 (127.17.110.2:40847)
I20260807 18:14:57.669845 26546 ts_manager.cc:194] Re-registered known tserver with Master: 480a54d1b82d42acb4428942491d0910 (127.17.110.2:40847)
I20260807 18:14:57.671653 26546 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.110.2@KRBTEST.COM'} at 127.17.110.2:42233
I20260807 18:14:57.764024 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 27021
I20260807 18:14:57.793498 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.3:37809
--local_ip_for_outbound_sockets=127.17.110.3
--tserver_master_addrs=127.17.110.62:40597
--webserver_port=34993
--webserver_interface=127.17.110.3
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.3.keytab
--principal=oryx/127.17.110.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:43781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.110.62:40597,127.17.110.61:37317
--tserver_master_addrs=127.17.110.62:40597,127.17.110.61:37317,127.17.110.60:34151 with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:14:57.909237 27931 heartbeater.cc:503] Master 127.17.110.62:40597 was elected leader, sending a full tablet report...
W20260807 18:14:58.231072 28149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:58.231441 28149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:58.231550 28149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:58.239557 28149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:58.239712 28149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:58.239847 28149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.3
I20260807 18:14:58.250525 28149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:43781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.3.keytab
--principal=oryx/127.17.110.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.3:37809
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.110.3
--webserver_port=34993
--tserver_master_addrs=127.17.110.62:40597,127.17.110.61:37317,127.17.110.60:34151
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.3
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:14:58.252386 28149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:58.254420 28149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:58.272100 28160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:58.274384 28158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:58.274907 28164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:58.276129 28149 server_base.cc:1034] running on GCE node
I20260807 18:14:58.277070 28149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:58.278373 28149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:58.279613 28149 hybrid_clock.cc:648] HybridClock initialized: now 1786126498279575 us; error 94 us; skew 500 ppm
I20260807 18:14:58.280043 28149 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:58 dist-test-slave-39l8 krb5kdc[26475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126498, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:58.284035 28149 init.cc:377] Logged in from keytab as oryx/127.17.110.3@KRBTEST.COM (short username oryx)
I20260807 18:14:58.285948 28149 webserver.cc:460] Webserver started at http://127.17.110.3:34993/ using document root <none> and password file <none>
I20260807 18:14:58.286832 28149 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:58.286980 28149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:58.293819 28149 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:14:58.297863 28173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:58.299916 28149 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 18:14:58.300117 28149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-2/wal
uuid: "25c40cc9663e47c8b30a95e33060db88"
format_stamp: "Formatted at 2026-08-07 18:14:54 on dist-test-slave-39l8"
server_key: "e6aecb5c7dd87543d7a05d55c302f74e"
server_key_iv: "9f13677d63f8d19b322f91a014714f2d"
server_key_version: "encryptionkey@0"
I20260807 18:14:58.301301 28149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:58.327217 28149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:58.332820 28149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:58.333097 28149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:58.334302 28149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:14:58.336043 28149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:14:58.336153 28149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:58.336236 28149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:14:58.336297 28149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:58.382026 28149 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.3:37809
I20260807 18:14:58.382169 28320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.3:37809 every 8 connection(s)
I20260807 18:14:58.383966 28149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
I20260807 18:14:58.392293 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 28149
Aug 07 18:14:58 dist-test-slave-39l8 krb5kdc[26475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126498, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.60@KRBTEST.COM
Aug 07 18:14:58 dist-test-slave-39l8 krb5kdc[26475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126498, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.61@KRBTEST.COM
Aug 07 18:14:58 dist-test-slave-39l8 krb5kdc[26475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126498, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM
I20260807 18:14:58.450004 28335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:58.388791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:34151 (local address 127.17.110.3:43973)
0807 18:14:58.399018 (+ 10227us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:58.399041 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:58.399213 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:58.401356 (+ 2143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:58.401378 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:58.402782 (+ 1404us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:58.404393 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:58.404451 (+ 58us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.406855 (+ 2404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.406865 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:58.410108 (+ 3243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:58.410144 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.415642 (+ 5498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.415655 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:58.416038 (+ 383us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:58.417230 (+ 1192us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:58.417326 (+ 96us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:58.430880 (+ 13554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:58.442331 (+ 11451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:58.442344 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:58.442367 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:58.443694 (+ 1327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:58.444519 (+ 825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:58.444524 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:58.444529 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:58.444884 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:58.446329 (+ 1445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:58.446342 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:58.447006 (+ 664us) client_negotiation.cc:769] Sending connection context
0807 18:14:58.447442 (+ 436us) client_negotiation.cc:240] Negotiation successful
0807 18:14:58.448281 (+ 839us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9974,"thread_start_us":494,"threads_started":1}
I20260807 18:14:58.453428 28327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:58.388646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:43973 (local address 127.17.110.60:34151)
0807 18:14:58.400033 (+ 11387us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:58.400042 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:58.400074 (+ 32us) server_negotiation.cc:407] Connection header received
0807 18:14:58.400168 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:58.400177 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:58.400434 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:58.404787 (+ 4353us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:58.405111 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.406550 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.412037 (+ 5487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.415048 (+ 3011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.415234 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:58.432661 (+ 17427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:58.432669 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:58.432674 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:58.432728 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:58.438564 (+ 5836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:58.443906 (+ 5342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:58.443913 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:58.443917 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:58.444323 (+ 406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:58.445211 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:58.445222 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:58.445225 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:58.445913 (+ 688us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:58.446231 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:58.451081 (+ 4850us) server_negotiation.cc:299] Negotiation successful
0807 18:14:58.452960 (+ 1879us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11137,"thread_start_us":10923,"threads_started":1}
I20260807 18:14:58.455474 28321 heartbeater.cc:347] Connected to a master server at 127.17.110.60:34151
I20260807 18:14:58.455916 28321 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:58.457141 28321 heartbeater.cc:511] Master 127.17.110.60:34151 requested a full tablet report, sending...
I20260807 18:14:58.458195 28332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:58.388791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:37317 (local address 127.17.110.3:38913)
0807 18:14:58.389922 (+ 1131us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:58.389991 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:58.394197 (+ 4206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:58.411534 (+ 17337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:58.411540 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:58.412289 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:58.412654 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:58.412671 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.420144 (+ 7473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.420155 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:58.422299 (+ 2144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:58.422331 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.423990 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.424002 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:58.424282 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:58.425203 (+ 921us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:58.425238 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:58.428970 (+ 3732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:58.451642 (+ 22672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:58.451648 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:58.451653 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:58.452329 (+ 676us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:58.455635 (+ 3306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:58.455647 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:58.455651 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:58.455933 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:58.456868 (+ 935us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:58.456873 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:58.457140 (+ 267us) client_negotiation.cc:769] Sending connection context
0807 18:14:58.457308 (+ 168us) client_negotiation.cc:240] Negotiation successful
0807 18:14:58.457671 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":750,"spinlock_wait_cycles":3328,"thread_start_us":9651,"threads_started":1}
I20260807 18:14:58.459836 27673 ts_manager.cc:194] Registered new tserver with Master: 25c40cc9663e47c8b30a95e33060db88 (127.17.110.3:37809)
I20260807 18:14:58.460038 28342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:58.405675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.17.110.3:41463)
0807 18:14:58.416058 (+ 10383us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:58.416081 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:58.416229 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:58.430049 (+ 13820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:58.430056 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:58.431838 (+ 1782us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:58.432284 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:58.432305 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.437305 (+ 5000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.437316 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:58.439544 (+ 2228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:58.439575 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.442737 (+ 3162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.442747 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:58.442961 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:58.445993 (+ 3032us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:58.446033 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:58.449818 (+ 3785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:58.456339 (+ 6521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:58.456346 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:58.456351 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:58.457008 (+ 657us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:58.457601 (+ 593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:58.457608 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:58.457613 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:58.457832 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:58.458762 (+ 930us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:58.458769 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:58.458952 (+ 183us) client_negotiation.cc:769] Sending connection context
0807 18:14:58.459122 (+ 170us) client_negotiation.cc:240] Negotiation successful
0807 18:14:58.459467 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10238,"thread_start_us":9903,"threads_started":1}
I20260807 18:14:58.460271 28326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:58.388140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:38913 (local address 127.17.110.61:37317)
0807 18:14:58.410992 (+ 22852us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:58.411005 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:58.411039 (+ 34us) server_negotiation.cc:407] Connection header received
0807 18:14:58.411134 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:58.411144 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:58.411268 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:58.411438 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:58.413421 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.419840 (+ 6419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.422656 (+ 2816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.423681 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.427007 (+ 3326us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:58.429322 (+ 2315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:58.429330 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:58.429335 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:58.429379 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:58.442021 (+ 12642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:58.452599 (+ 10578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:58.452604 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:58.452608 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:58.452826 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:58.456185 (+ 3359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:58.456191 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:58.456195 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:58.456682 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:58.459227 (+ 2545us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:58.459406 (+ 179us) server_negotiation.cc:299] Negotiation successful
0807 18:14:58.459767 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22513,"thread_start_us":22357,"threads_started":1}
I20260807 18:14:58.461228 28346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:58.399498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:41463 (local address 127.17.110.62:40597)
0807 18:14:58.413574 (+ 14076us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:58.413583 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:58.422798 (+ 9215us) server_negotiation.cc:407] Connection header received
0807 18:14:58.429581 (+ 6783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:58.429591 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:58.429757 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:58.429929 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:58.433315 (+ 3386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.435040 (+ 1725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.440037 (+ 4997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.440819 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.440979 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:58.450142 (+ 9163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:58.450153 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:58.450160 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:58.450227 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:58.455654 (+ 5427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:58.457234 (+ 1580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:58.457243 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:58.457248 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:58.457441 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:58.457995 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:58.458002 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:58.458007 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:58.458461 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:58.458642 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:58.460465 (+ 1823us) server_negotiation.cc:299] Negotiation successful
0807 18:14:58.460803 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13886,"thread_start_us":37524,"threads_started":1}
I20260807 18:14:58.461387 28323 heartbeater.cc:347] Connected to a master server at 127.17.110.61:37317
I20260807 18:14:58.461552 28323 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:58.462091 28323 heartbeater.cc:511] Master 127.17.110.61:37317 requested a full tablet report, sending...
I20260807 18:14:58.462859 28322 heartbeater.cc:347] Connected to a master server at 127.17.110.62:40597
I20260807 18:14:58.463001 28322 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:58.463608 28322 heartbeater.cc:511] Master 127.17.110.62:40597 requested a full tablet report, sending...
I20260807 18:14:58.466042 27394 ts_manager.cc:194] Registered new tserver with Master: 25c40cc9663e47c8b30a95e33060db88 (127.17.110.3:37809)
I20260807 18:14:58.466991 26546 ts_manager.cc:194] Re-registered known tserver with Master: 25c40cc9663e47c8b30a95e33060db88 (127.17.110.3:37809)
I20260807 18:14:58.468076 28350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:58.416535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46920 (local address 127.17.110.60:34151)
0807 18:14:58.429280 (+ 12745us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:58.429287 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:58.441596 (+ 12309us) server_negotiation.cc:407] Connection header received
0807 18:14:58.441953 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:58.441959 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:58.442064 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:58.442160 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:58.445171 (+ 3011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.446916 (+ 1745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.451209 (+ 4293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.452073 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.452204 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:58.457493 (+ 5289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:58.457504 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:58.457511 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:58.457564 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:58.462355 (+ 4791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:58.463559 (+ 1204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:58.463567 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:58.463573 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:58.463786 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:58.464505 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:58.464511 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:58.464515 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:58.465107 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:58.465388 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:58.467353 (+ 1965us) server_negotiation.cc:299] Negotiation successful
0807 18:14:58.467660 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12552,"thread_start_us":12761,"threads_started":1}
I20260807 18:14:58.468266 26546 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.110.3@KRBTEST.COM'} at 127.17.110.3:41463
I20260807 18:14:58.476082 28340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:58.412889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58370 (local address 127.17.110.62:40597)
0807 18:14:58.429925 (+ 17036us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:58.429932 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:58.432470 (+ 2538us) server_negotiation.cc:407] Connection header received
0807 18:14:58.432555 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:58.432560 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:58.432664 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:58.432784 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:58.440590 (+ 7806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.442426 (+ 1836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.447939 (+ 5513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.448672 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.448804 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:58.466164 (+ 17360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:58.466172 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:58.466178 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:58.466228 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:58.470791 (+ 4563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:58.471943 (+ 1152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:58.471950 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:58.471953 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:58.472170 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:58.473143 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:58.473153 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:58.473157 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:58.473846 (+ 689us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:58.474238 (+ 392us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:58.475054 (+ 816us) server_negotiation.cc:299] Negotiation successful
0807 18:14:58.475452 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16878,"thread_start_us":305,"threads_started":1}
I20260807 18:14:58.484457 28353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:58.432150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47458 (local address 127.17.110.61:37317)
0807 18:14:58.440408 (+ 8258us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:58.440417 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:58.452053 (+ 11636us) server_negotiation.cc:407] Connection header received
0807 18:14:58.452136 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:58.452141 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:58.452252 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:58.452385 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:58.463457 (+ 11072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.468093 (+ 4636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.470616 (+ 2523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.471390 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.471546 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:58.476678 (+ 5132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:58.476685 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:58.476689 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:58.476759 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:58.480672 (+ 3913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:58.481605 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:58.481612 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:58.481615 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:58.481833 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:58.482394 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:58.482400 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:58.482405 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:58.482862 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:58.483124 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:58.483690 (+ 566us) server_negotiation.cc:299] Negotiation successful
0807 18:14:58.483979 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8002,"thread_start_us":9028,"threads_started":1}
I20260807 18:14:58.482015 26546 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58370:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:14:58.491866 26546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:14:58.545043 28369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:58.519186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:40847 (local address 127.0.0.1:58638)
0807 18:14:58.535321 (+ 16135us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:58.535345 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:58.535484 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:58.536212 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:58.536216 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:58.536243 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:14:58.536778 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:58.536800 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.539519 (+ 2719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.539525 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:58.542594 (+ 3069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:58.542607 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.544051 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.544058 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:58.544242 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:58.544285 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 18:14:58.544402 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 18:14:58.544562 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15877,"mutex_wait_us":1,"thread_start_us":15719,"threads_started":1}
I20260807 18:14:58.545569 28366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:58.518393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:33421 (local address 127.0.0.1:56358)
0807 18:14:58.534279 (+ 15886us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:58.534300 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:58.534547 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:58.535595 (+ 1048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:58.535603 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:58.535644 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:14:58.536227 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:58.536245 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.538757 (+ 2512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.538765 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:58.541891 (+ 3126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:58.541916 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.544408 (+ 2492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.544415 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:58.544587 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:58.544619 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 18:14:58.544709 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 18:14:58.544851 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15733,"mutex_wait_us":2,"thread_start_us":15592,"threads_started":1}
I20260807 18:14:58.545878 28371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:58.519581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58638 (local address 127.17.110.2:40847)
0807 18:14:58.533628 (+ 14047us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:58.533637 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:58.535631 (+ 1994us) server_negotiation.cc:407] Connection header received
0807 18:14:58.535757 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:58.535766 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:58.535997 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:58.536218 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:14:58.537003 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.539143 (+ 2140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.542824 (+ 3681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.543786 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.543965 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:58.544248 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:58.545353 (+ 1105us) server_negotiation.cc:299] Negotiation successful
0807 18:14:58.545574 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13872,"thread_start_us":14057,"threads_started":1}
I20260807 18:14:58.547303 28377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:58.519971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50264 (local address 127.17.110.3:37809)
0807 18:14:58.533356 (+ 13385us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:58.533370 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:58.537140 (+ 3770us) server_negotiation.cc:407] Connection header received
0807 18:14:58.537289 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:58.537297 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:58.537561 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:58.537795 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:14:58.538812 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.541625 (+ 2813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.544403 (+ 2778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.546040 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.546198 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:58.546465 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:58.546743 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 18:14:58.546941 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13104,"thread_start_us":13413,"threads_started":1}
I20260807 18:14:58.547358 28373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:58.519725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:37809 (local address 127.0.0.1:50264)
0807 18:14:58.536969 (+ 17244us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:58.536995 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:58.537130 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:58.537817 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:58.537822 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:58.537850 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:14:58.538278 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:58.538290 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.541832 (+ 3542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.541839 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:58.544113 (+ 2274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:58.544171 (+ 58us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.546291 (+ 2120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.546297 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:58.546503 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:58.546539 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 18:14:58.546644 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 18:14:58.546830 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17079,"thread_start_us":16827,"threads_started":1}
I20260807 18:14:58.547401 28368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:58.518952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56358 (local address 127.17.110.1:33421)
0807 18:14:58.526513 (+ 7561us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:58.526529 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:58.534487 (+ 7958us) server_negotiation.cc:407] Connection header received
0807 18:14:58.534838 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:58.534851 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:58.535330 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:58.535596 (+ 266us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:14:58.537077 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.538533 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.542152 (+ 3619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.543858 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.544018 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:58.544284 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:58.546596 (+ 2312us) server_negotiation.cc:299] Negotiation successful
0807 18:14:58.546903 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7315,"thread_start_us":7151,"threads_started":1}
I20260807 18:14:58.551553 28043 tablet_service.cc:1467] Processing CreateTablet for tablet e65647e8b66c4db4a46e43e621352863 (DEFAULT_TABLE table=test-table [id=9d7e8661a8014d169e162bb6e4e1d7a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:14:58.552973 27836 tablet_service.cc:1467] Processing CreateTablet for tablet e65647e8b66c4db4a46e43e621352863 (DEFAULT_TABLE table=test-table [id=9d7e8661a8014d169e162bb6e4e1d7a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:14:58.553872 28043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e65647e8b66c4db4a46e43e621352863. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:58.554350 27836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e65647e8b66c4db4a46e43e621352863. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:58.554370 28232 tablet_service.cc:1467] Processing CreateTablet for tablet e65647e8b66c4db4a46e43e621352863 (DEFAULT_TABLE table=test-table [id=9d7e8661a8014d169e162bb6e4e1d7a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:14:58.556660 28232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e65647e8b66c4db4a46e43e621352863. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:58.570585 28379 tablet_bootstrap.cc:492] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490: Bootstrap starting.
I20260807 18:14:58.574802 28379 tablet_bootstrap.cc:654] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:58.575635 28380 tablet_bootstrap.cc:492] T e65647e8b66c4db4a46e43e621352863 P 480a54d1b82d42acb4428942491d0910: Bootstrap starting.
I20260807 18:14:58.578171 28379 log.cc:824] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490: Log is configured to *not* fsync() on all Append() calls
I20260807 18:14:58.580032 28381 tablet_bootstrap.cc:492] T e65647e8b66c4db4a46e43e621352863 P 25c40cc9663e47c8b30a95e33060db88: Bootstrap starting.
I20260807 18:14:58.581030 28380 tablet_bootstrap.cc:654] T e65647e8b66c4db4a46e43e621352863 P 480a54d1b82d42acb4428942491d0910: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:58.582726 28380 log.cc:824] T e65647e8b66c4db4a46e43e621352863 P 480a54d1b82d42acb4428942491d0910: Log is configured to *not* fsync() on all Append() calls
I20260807 18:14:58.583847 28379 tablet_bootstrap.cc:492] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490: No bootstrap required, opened a new log
I20260807 18:14:58.584208 28379 ts_tablet_manager.cc:1397] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.000s
I20260807 18:14:58.584582 28381 tablet_bootstrap.cc:654] T e65647e8b66c4db4a46e43e621352863 P 25c40cc9663e47c8b30a95e33060db88: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:58.586802 28381 log.cc:824] T e65647e8b66c4db4a46e43e621352863 P 25c40cc9663e47c8b30a95e33060db88: Log is configured to *not* fsync() on all Append() calls
I20260807 18:14:58.587391 28380 tablet_bootstrap.cc:492] T e65647e8b66c4db4a46e43e621352863 P 480a54d1b82d42acb4428942491d0910: No bootstrap required, opened a new log
I20260807 18:14:58.587872 28380 ts_tablet_manager.cc:1397] T e65647e8b66c4db4a46e43e621352863 P 480a54d1b82d42acb4428942491d0910: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.002s
I20260807 18:14:58.591692 28381 tablet_bootstrap.cc:492] T e65647e8b66c4db4a46e43e621352863 P 25c40cc9663e47c8b30a95e33060db88: No bootstrap required, opened a new log
I20260807 18:14:58.592134 28381 ts_tablet_manager.cc:1397] T e65647e8b66c4db4a46e43e621352863 P 25c40cc9663e47c8b30a95e33060db88: Time spent bootstrapping tablet: real 0.013s user 0.011s sys 0.000s
I20260807 18:14:58.597772 28379 raft_consensus.cc:348] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6fda665629f44b69b25453b35fb9490" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 33421 } } peers { permanent_uuid: "480a54d1b82d42acb4428942491d0910" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40847 } } peers { permanent_uuid: "25c40cc9663e47c8b30a95e33060db88" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37809 } }
I20260807 18:14:58.598381 28379 raft_consensus.cc:374] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:58.598506 28379 raft_consensus.cc:729] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6fda665629f44b69b25453b35fb9490, State: Initialized, Role: FOLLOWER
I20260807 18:14:58.599473 28379 consensus_queue.cc:260] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6fda665629f44b69b25453b35fb9490" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 33421 } } peers { permanent_uuid: "480a54d1b82d42acb4428942491d0910" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40847 } } peers { permanent_uuid: "25c40cc9663e47c8b30a95e33060db88" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37809 } }
I20260807 18:14:58.603811 28380 raft_consensus.cc:348] T e65647e8b66c4db4a46e43e621352863 P 480a54d1b82d42acb4428942491d0910 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6fda665629f44b69b25453b35fb9490" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 33421 } } peers { permanent_uuid: "480a54d1b82d42acb4428942491d0910" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40847 } } peers { permanent_uuid: "25c40cc9663e47c8b30a95e33060db88" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37809 } }
I20260807 18:14:58.604540 28380 raft_consensus.cc:374] T e65647e8b66c4db4a46e43e621352863 P 480a54d1b82d42acb4428942491d0910 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:58.604683 28380 raft_consensus.cc:729] T e65647e8b66c4db4a46e43e621352863 P 480a54d1b82d42acb4428942491d0910 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 480a54d1b82d42acb4428942491d0910, State: Initialized, Role: FOLLOWER
I20260807 18:14:58.605796 28380 consensus_queue.cc:260] T e65647e8b66c4db4a46e43e621352863 P 480a54d1b82d42acb4428942491d0910 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6fda665629f44b69b25453b35fb9490" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 33421 } } peers { permanent_uuid: "480a54d1b82d42acb4428942491d0910" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40847 } } peers { permanent_uuid: "25c40cc9663e47c8b30a95e33060db88" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37809 } }
I20260807 18:14:58.608346 28379 ts_tablet_manager.cc:1428] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490: Time spent starting tablet: real 0.024s user 0.019s sys 0.005s
W20260807 18:14:58.615067 27937 tablet.cc:2369] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:14:58.621335 28125 heartbeater.cc:503] Master 127.17.110.62:40597 was elected leader, sending a full tablet report...
I20260807 18:14:58.622290 28380 ts_tablet_manager.cc:1428] T e65647e8b66c4db4a46e43e621352863 P 480a54d1b82d42acb4428942491d0910: Time spent starting tablet: real 0.034s user 0.020s sys 0.008s
I20260807 18:14:58.623240 28381 raft_consensus.cc:348] T e65647e8b66c4db4a46e43e621352863 P 25c40cc9663e47c8b30a95e33060db88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6fda665629f44b69b25453b35fb9490" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 33421 } } peers { permanent_uuid: "480a54d1b82d42acb4428942491d0910" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40847 } } peers { permanent_uuid: "25c40cc9663e47c8b30a95e33060db88" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37809 } }
I20260807 18:14:58.623874 28381 raft_consensus.cc:374] T e65647e8b66c4db4a46e43e621352863 P 25c40cc9663e47c8b30a95e33060db88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:58.624001 28381 raft_consensus.cc:729] T e65647e8b66c4db4a46e43e621352863 P 25c40cc9663e47c8b30a95e33060db88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25c40cc9663e47c8b30a95e33060db88, State: Initialized, Role: FOLLOWER
I20260807 18:14:58.624925 28381 consensus_queue.cc:260] T e65647e8b66c4db4a46e43e621352863 P 25c40cc9663e47c8b30a95e33060db88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6fda665629f44b69b25453b35fb9490" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 33421 } } peers { permanent_uuid: "480a54d1b82d42acb4428942491d0910" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40847 } } peers { permanent_uuid: "25c40cc9663e47c8b30a95e33060db88" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37809 } }
I20260807 18:14:58.629010 28381 ts_tablet_manager.cc:1428] T e65647e8b66c4db4a46e43e621352863 P 25c40cc9663e47c8b30a95e33060db88: Time spent starting tablet: real 0.037s user 0.016s sys 0.003s
I20260807 18:14:58.629412 28322 heartbeater.cc:503] Master 127.17.110.62:40597 was elected leader, sending a full tablet report...
W20260807 18:14:58.643247 28324 tablet.cc:2369] T e65647e8b66c4db4a46e43e621352863 P 25c40cc9663e47c8b30a95e33060db88: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:14:58.734117 28385 raft_consensus.cc:482] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:14:58.734431 28385 raft_consensus.cc:504] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6fda665629f44b69b25453b35fb9490" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 33421 } } peers { permanent_uuid: "480a54d1b82d42acb4428942491d0910" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40847 } } peers { permanent_uuid: "25c40cc9663e47c8b30a95e33060db88" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37809 } }
I20260807 18:14:58.736574 28385 leader_election.cc:288] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 480a54d1b82d42acb4428942491d0910 (127.17.110.2:40847), 25c40cc9663e47c8b30a95e33060db88 (127.17.110.3:37809)
I20260807 18:14:58.743981 28377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:58.737126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:52467 (local address 127.17.110.3:37809)
0807 18:14:58.737571 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:58.737577 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:58.738179 (+ 602us) server_negotiation.cc:407] Connection header received
0807 18:14:58.738370 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:58.738374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:58.738468 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:58.738565 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:14:58.739343 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.740133 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.742098 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.742964 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.743147 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:58.743261 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:58.743573 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 18:14:58.743704 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286}
I20260807 18:14:58.744024 28371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:58.737036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:39309 (local address 127.17.110.2:40847)
0807 18:14:58.737363 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:58.737374 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:58.738057 (+ 683us) server_negotiation.cc:407] Connection header received
0807 18:14:58.738264 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:58.738272 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:58.738367 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:58.738438 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:14:58.739210 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.739981 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.742065 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.742971 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.743184 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:58.743264 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:58.743616 (+ 352us) server_negotiation.cc:299] Negotiation successful
0807 18:14:58.743750 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 18:14:58.744308 28389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:58.736723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:40847 (local address 127.17.110.1:39309)
0807 18:14:58.737908 (+ 1185us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:58.737929 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:58.738077 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:58.738487 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:58.738493 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:58.738523 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:14:58.739024 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:58.739034 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.740133 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.740140 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:58.741867 (+ 1727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:58.741882 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.743278 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.743289 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:58.743419 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:58.743443 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 18:14:58.743560 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 18:14:58.743704 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":999,"mutex_wait_us":8,"thread_start_us":1024,"threads_started":1}
I20260807 18:14:58.744469 28388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:58.736938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:37809 (local address 127.17.110.1:52467)
0807 18:14:58.738031 (+ 1093us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:58.738057 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:58.738233 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:58.738616 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:58.738619 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:58.738645 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:14:58.739137 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:58.739150 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.740279 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.740283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:58.741883 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:58.741897 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.743222 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.743230 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:58.743397 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:58.743424 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 18:14:58.743507 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 18:14:58.743674 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":959,"mutex_wait_us":86,"thread_start_us":499,"threads_started":1}
I20260807 18:14:58.745647 28068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e65647e8b66c4db4a46e43e621352863" candidate_uuid: "d6fda665629f44b69b25453b35fb9490" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "480a54d1b82d42acb4428942491d0910" is_pre_election: true
I20260807 18:14:58.745647 28263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e65647e8b66c4db4a46e43e621352863" candidate_uuid: "d6fda665629f44b69b25453b35fb9490" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25c40cc9663e47c8b30a95e33060db88" is_pre_election: true
I20260807 18:14:58.746294 28263 raft_consensus.cc:2454] T e65647e8b66c4db4a46e43e621352863 P 25c40cc9663e47c8b30a95e33060db88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6fda665629f44b69b25453b35fb9490 in term 0.
I20260807 18:14:58.746294 28068 raft_consensus.cc:2454] T e65647e8b66c4db4a46e43e621352863 P 480a54d1b82d42acb4428942491d0910 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6fda665629f44b69b25453b35fb9490 in term 0.
I20260807 18:14:58.747149 27773 leader_election.cc:302] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 25c40cc9663e47c8b30a95e33060db88, 480a54d1b82d42acb4428942491d0910, d6fda665629f44b69b25453b35fb9490; no voters:
I20260807 18:14:58.747802 28385 raft_consensus.cc:2781] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:14:58.747983 28385 raft_consensus.cc:482] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:14:58.748100 28385 raft_consensus.cc:3037] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:58.750459 28385 raft_consensus.cc:504] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6fda665629f44b69b25453b35fb9490" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 33421 } } peers { permanent_uuid: "480a54d1b82d42acb4428942491d0910" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40847 } } peers { permanent_uuid: "25c40cc9663e47c8b30a95e33060db88" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37809 } }
I20260807 18:14:58.751256 28385 leader_election.cc:288] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490 [CANDIDATE]: Term 1 election: Requested vote from peers 480a54d1b82d42acb4428942491d0910 (127.17.110.2:40847), 25c40cc9663e47c8b30a95e33060db88 (127.17.110.3:37809)
I20260807 18:14:58.751866 28068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e65647e8b66c4db4a46e43e621352863" candidate_uuid: "d6fda665629f44b69b25453b35fb9490" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "480a54d1b82d42acb4428942491d0910"
I20260807 18:14:58.751909 28263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e65647e8b66c4db4a46e43e621352863" candidate_uuid: "d6fda665629f44b69b25453b35fb9490" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25c40cc9663e47c8b30a95e33060db88"
I20260807 18:14:58.752133 28068 raft_consensus.cc:3037] T e65647e8b66c4db4a46e43e621352863 P 480a54d1b82d42acb4428942491d0910 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:58.752146 28263 raft_consensus.cc:3037] T e65647e8b66c4db4a46e43e621352863 P 25c40cc9663e47c8b30a95e33060db88 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:58.754668 28068 raft_consensus.cc:2454] T e65647e8b66c4db4a46e43e621352863 P 480a54d1b82d42acb4428942491d0910 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6fda665629f44b69b25453b35fb9490 in term 1.
I20260807 18:14:58.754666 28263 raft_consensus.cc:2454] T e65647e8b66c4db4a46e43e621352863 P 25c40cc9663e47c8b30a95e33060db88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6fda665629f44b69b25453b35fb9490 in term 1.
I20260807 18:14:58.755347 27771 leader_election.cc:302] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 25c40cc9663e47c8b30a95e33060db88, 480a54d1b82d42acb4428942491d0910, d6fda665629f44b69b25453b35fb9490; no voters:
I20260807 18:14:58.755741 28385 raft_consensus.cc:2781] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:14:58.756238 28385 raft_consensus.cc:686] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490 [term 1 LEADER]: Becoming Leader. State: Replica: d6fda665629f44b69b25453b35fb9490, State: Running, Role: LEADER
I20260807 18:14:58.756839 28385 consensus_queue.cc:237] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6fda665629f44b69b25453b35fb9490" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 33421 } } peers { permanent_uuid: "480a54d1b82d42acb4428942491d0910" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40847 } } peers { permanent_uuid: "25c40cc9663e47c8b30a95e33060db88" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37809 } }
I20260807 18:14:58.762768 26544 catalog_manager.cc:5495] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6fda665629f44b69b25453b35fb9490 (127.17.110.1). New cstate: current_term: 1 leader_uuid: "d6fda665629f44b69b25453b35fb9490" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6fda665629f44b69b25453b35fb9490" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 33421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "480a54d1b82d42acb4428942491d0910" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25c40cc9663e47c8b30a95e33060db88" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37809 } health_report { overall_health: UNKNOWN } } }
I20260807 18:14:58.804363 28368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:58.795773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56368 (local address 127.17.110.1:33421)
0807 18:14:58.795998 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:58.796007 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:58.796100 (+ 93us) server_negotiation.cc:407] Connection header received
0807 18:14:58.796394 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:58.796398 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:58.796524 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:58.796780 (+ 256us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:14:58.797401 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.798665 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.800819 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.801602 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.801778 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:58.802482 (+ 704us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:14:58.803573 (+ 1091us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:14:58.803762 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:58.803888 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 18:14:58.804000 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 18:14:58.816255 28263 raft_consensus.cc:1261] T e65647e8b66c4db4a46e43e621352863 P 25c40cc9663e47c8b30a95e33060db88 [term 1 FOLLOWER]: Refusing update from remote peer d6fda665629f44b69b25453b35fb9490: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:14:58.816624 28068 raft_consensus.cc:1261] T e65647e8b66c4db4a46e43e621352863 P 480a54d1b82d42acb4428942491d0910 [term 1 FOLLOWER]: Refusing update from remote peer d6fda665629f44b69b25453b35fb9490: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:14:58.817338 28385 consensus_queue.cc:1035] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25c40cc9663e47c8b30a95e33060db88" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:14:58.817765 28390 consensus_queue.cc:1035] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490 [LEADER]: Connected to new peer: Peer: permanent_uuid: "480a54d1b82d42acb4428942491d0910" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:14:58.835731 28393 mvcc.cc:205] Tried to move back new op lower bound from 7315974139138113536 to 7315974138911674368. Current Snapshot: MvccSnapshot[applied={T|T < 7315974139138113536}]
I20260807 18:14:58.838289 28394 mvcc.cc:205] Tried to move back new op lower bound from 7315974139138113536 to 7315974138911674368. Current Snapshot: MvccSnapshot[applied={T|T < 7315974139138113536}]
W20260807 18:14:58.849390 28128 tablet.cc:2369] T e65647e8b66c4db4a46e43e621352863 P 480a54d1b82d42acb4428942491d0910: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:14:58.870301 26544 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:14:58.870639 26544 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:14:58.878647 26544 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ce9e1ae9d6354f9d9b7020b5e73f2ef2: Sending DeleteTablet for 3 replicas of tablet e65647e8b66c4db4a46e43e621352863
I20260807 18:14:58.880116 27836 tablet_service.cc:1514] Processing DeleteTablet for tablet e65647e8b66c4db4a46e43e621352863 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:14:58 UTC) from {username='oryx', principal='oryx/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:56358
I20260807 18:14:58.880334 28043 tablet_service.cc:1514] Processing DeleteTablet for tablet e65647e8b66c4db4a46e43e621352863 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:14:58 UTC) from {username='oryx', principal='oryx/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:58638
I20260807 18:14:58.880616 28232 tablet_service.cc:1514] Processing DeleteTablet for tablet e65647e8b66c4db4a46e43e621352863 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:14:58 UTC) from {username='oryx', principal='oryx/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:50264
I20260807 18:14:58.892145 28404 tablet_replica.cc:330] T e65647e8b66c4db4a46e43e621352863 P 480a54d1b82d42acb4428942491d0910: stopping tablet replica
I20260807 18:14:58.892865 28404 raft_consensus.cc:2229] T e65647e8b66c4db4a46e43e621352863 P 480a54d1b82d42acb4428942491d0910 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:14:58.893541 28404 raft_consensus.cc:2258] T e65647e8b66c4db4a46e43e621352863 P 480a54d1b82d42acb4428942491d0910 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:14:58.893592 28403 tablet_replica.cc:330] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490: stopping tablet replica
I20260807 18:14:58.894522 28403 raft_consensus.cc:2229] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:14:58.895702 28403 raft_consensus.cc:2258] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:14:58.895820 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 26498
I20260807 18:14:58.896590 28407 tablet_replica.cc:330] T e65647e8b66c4db4a46e43e621352863 P 25c40cc9663e47c8b30a95e33060db88: stopping tablet replica
I20260807 18:14:58.897035 26589 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:14:58.897032 28407 raft_consensus.cc:2229] T e65647e8b66c4db4a46e43e621352863 P 25c40cc9663e47c8b30a95e33060db88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:14:58.897285 28404 ts_tablet_manager.cc:1905] T e65647e8b66c4db4a46e43e621352863 P 480a54d1b82d42acb4428942491d0910: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:14:58.897575 28407 raft_consensus.cc:2258] T e65647e8b66c4db4a46e43e621352863 P 25c40cc9663e47c8b30a95e33060db88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:14:58.898568 28403 ts_tablet_manager.cc:1905] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:14:58.899641 28407 ts_tablet_manager.cc:1905] T e65647e8b66c4db4a46e43e621352863 P 25c40cc9663e47c8b30a95e33060db88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:14:58.907083 28404 ts_tablet_manager.cc:1918] T e65647e8b66c4db4a46e43e621352863 P 480a54d1b82d42acb4428942491d0910: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:14:58.907368 28407 ts_tablet_manager.cc:1918] T e65647e8b66c4db4a46e43e621352863 P 25c40cc9663e47c8b30a95e33060db88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:14:58.907603 28407 log.cc:1196] T e65647e8b66c4db4a46e43e621352863 P 25c40cc9663e47c8b30a95e33060db88: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-2/wal/wals/e65647e8b66c4db4a46e43e621352863
I20260807 18:14:58.908174 28404 log.cc:1196] T e65647e8b66c4db4a46e43e621352863 P 480a54d1b82d42acb4428942491d0910: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-1/wal/wals/e65647e8b66c4db4a46e43e621352863
I20260807 18:14:58.908509 28407 ts_tablet_manager.cc:1939] T e65647e8b66c4db4a46e43e621352863 P 25c40cc9663e47c8b30a95e33060db88: Deleting consensus metadata
I20260807 18:14:58.909153 28404 ts_tablet_manager.cc:1939] T e65647e8b66c4db4a46e43e621352863 P 480a54d1b82d42acb4428942491d0910: Deleting consensus metadata
I20260807 18:14:58.913398 28403 ts_tablet_manager.cc:1918] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:14:58.913643 28403 log.cc:1196] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-0/wal/wals/e65647e8b66c4db4a46e43e621352863
I20260807 18:14:58.914291 28403 ts_tablet_manager.cc:1939] T e65647e8b66c4db4a46e43e621352863 P d6fda665629f44b69b25453b35fb9490: Deleting consensus metadata
I20260807 18:14:58.924522 28353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:58.883091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47470 (local address 127.17.110.61:37317)
0807 18:14:58.883322 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:58.883331 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:58.893856 (+ 10525us) server_negotiation.cc:407] Connection header received
0807 18:14:58.895796 (+ 1940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:58.895804 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:58.895904 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:58.895994 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:58.908182 (+ 12188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.909349 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.911779 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.912501 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.912652 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:58.916936 (+ 4284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:58.916944 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:58.916949 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:58.916998 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:58.920814 (+ 3816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:58.921818 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:58.921824 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:58.921828 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:58.921995 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:58.922460 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:58.922466 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:58.922469 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:58.922951 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:58.923227 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:58.923880 (+ 653us) server_negotiation.cc:299] Negotiation successful
0807 18:14:58.924158 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 18:14:58.944777 28350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:58.894811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46932 (local address 127.17.110.60:34151)
0807 18:14:58.895021 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:58.895027 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:58.925992 (+ 30965us) server_negotiation.cc:407] Connection header received
0807 18:14:58.926204 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:58.926208 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:58.926325 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:58.926432 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:58.929198 (+ 2766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.930348 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.932401 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:58.933142 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:58.933297 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:58.937239 (+ 3942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:58.937248 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:58.937254 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:58.937332 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:58.941274 (+ 3942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:58.942295 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:58.942301 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:58.942305 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:58.942471 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:58.942954 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:58.942960 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:58.942964 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:58.943336 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:58.943593 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:58.943983 (+ 390us) server_negotiation.cc:299] Negotiation successful
0807 18:14:58.944234 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 18:14:59.041077 26521 catalog_manager.cc:4841] TS 480a54d1b82d42acb4428942491d0910 (127.17.110.2:40847): tablet e65647e8b66c4db4a46e43e621352863 (table test-table [id=9d7e8661a8014d169e162bb6e4e1d7a8]) successfully deleted
I20260807 18:14:59.041144 26522 catalog_manager.cc:4841] TS d6fda665629f44b69b25453b35fb9490 (127.17.110.1:33421): tablet e65647e8b66c4db4a46e43e621352863 (table test-table [id=9d7e8661a8014d169e162bb6e4e1d7a8]) successfully deleted
I20260807 18:14:59.043411 26518 catalog_manager.cc:4841] TS 25c40cc9663e47c8b30a95e33060db88 (127.17.110.3:37809): tablet e65647e8b66c4db4a46e43e621352863 (table test-table [id=9d7e8661a8014d169e162bb6e4e1d7a8]) successfully deleted
I20260807 18:14:59.044217 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 26498
I20260807 18:14:59.109581 28350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:59.084308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46942 (local address 127.17.110.60:34151)
0807 18:14:59.084572 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:59.084581 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:59.086367 (+ 1786us) server_negotiation.cc:407] Connection header received
0807 18:14:59.086740 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:59.086750 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:59.086872 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:59.086976 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:59.090322 (+ 3346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:59.091906 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:59.094864 (+ 2958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:59.095826 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:59.095974 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:59.101512 (+ 5538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:59.101524 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:59.101532 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:59.101603 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:59.106241 (+ 4638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:59.107149 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:59.107155 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:59.107159 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:59.107314 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:59.107755 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:59.107761 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:59.107764 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:59.108062 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:59.108336 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:59.108717 (+ 381us) server_negotiation.cc:299] Negotiation successful
0807 18:14:59.109316 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 18:14:59.121299 28353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:59.085363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47474 (local address 127.17.110.61:37317)
0807 18:14:59.085608 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:59.085616 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:59.105346 (+ 19730us) server_negotiation.cc:407] Connection header received
0807 18:14:59.105578 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:59.105587 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:59.105704 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:59.105823 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:59.108307 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:59.109646 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:59.111467 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:59.111939 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:59.112124 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:59.115203 (+ 3079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:59.115209 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:59.115213 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:59.115254 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:59.118307 (+ 3053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:59.119178 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:59.119184 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:59.119186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:59.119320 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:59.119722 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:59.119726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:59.119728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:59.120032 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:59.120325 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:59.120712 (+ 387us) server_negotiation.cc:299] Negotiation successful
0807 18:14:59.120946 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 18:14:59.857291 28424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:59.855109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.17.110.1:56685)
0807 18:14:59.856818 (+ 1709us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:59.857011 (+ 193us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1533,"thread_start_us":1381,"threads_started":1}
I20260807 18:14:59.857591 28423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:59.855049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.17.110.3:41137)
0807 18:14:59.857223 (+ 2174us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:59.857367 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2019,"thread_start_us":1901,"threads_started":1}
W20260807 18:14:59.858012 27931 heartbeater.cc:644] Failed to heartbeat to 127.17.110.62:40597 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
W20260807 18:14:59.858153 28322 heartbeater.cc:644] Failed to heartbeat to 127.17.110.62:40597 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
I20260807 18:14:59.861778 28426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:59.860959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.17.110.2:42245)
0807 18:14:59.861448 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:59.861565 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":239,"threads_started":1}
W20260807 18:14:59.862183 28125 heartbeater.cc:644] Failed to heartbeat to 127.17.110.62:40597 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
I20260807 18:15:00.378863 28427 raft_consensus.cc:482] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce9e1ae9d6354f9d9b7020b5e73f2ef2)
I20260807 18:15:00.379324 28427 raft_consensus.cc:504] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: false } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: false } }
I20260807 18:15:00.379462 28428 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce9e1ae9d6354f9d9b7020b5e73f2ef2)
I20260807 18:15:00.379758 28428 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: false } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: false } }
I20260807 18:15:00.380697 28428 leader_election.cc:288] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce9e1ae9d6354f9d9b7020b5e73f2ef2 (127.17.110.62:40597), 822d5b469a444f5ea371b43ecd44bb62 (127.17.110.60:34151)
I20260807 18:15:00.381017 28427 leader_election.cc:288] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce9e1ae9d6354f9d9b7020b5e73f2ef2 (127.17.110.62:40597), 94d47e0385a840749cd3676abfda32c0 (127.17.110.61:37317)
I20260807 18:15:00.390554 28431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.381003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.0.0.1:58772)
0807 18:15:00.390008 (+ 9005us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:00.390276 (+ 268us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":8863,"thread_start_us":8685,"threads_started":1}
W20260807 18:15:00.390951 27377 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
I20260807 18:15:00.391151 28432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.381079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.0.0.1:58776)
0807 18:15:00.390578 (+ 9499us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:00.390803 (+ 225us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":9349,"threads_started":1}
W20260807 18:15:00.392535 27649 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
I20260807 18:15:00.393218 28431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.392674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.0.0.1:58782)
0807 18:15:00.392891 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:00.393001 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260807 18:15:00.393649 27377 leader_election.cc:334] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ce9e1ae9d6354f9d9b7020b5e73f2ef2 (127.17.110.62:40597): Network error: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
I20260807 18:15:00.396121 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.394966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.0.0.1:58784)
0807 18:15:00.395544 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:00.395772 (+ 228us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":318,"threads_started":1}
W20260807 18:15:00.396368 27649 leader_election.cc:334] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ce9e1ae9d6354f9d9b7020b5e73f2ef2 (127.17.110.62:40597): Network error: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
I20260807 18:15:00.398403 28434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.382199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46948 (local address 127.17.110.60:34151)
0807 18:15:00.390383 (+ 8184us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:00.390389 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:00.390416 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:15:00.390507 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:00.390514 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:00.390611 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:00.390916 (+ 305us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:15:00.392084 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:00.393616 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:00.396238 (+ 2622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:00.397273 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:00.397432 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:00.397544 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:00.397967 (+ 423us) server_negotiation.cc:299] Negotiation successful
0807 18:15:00.398153 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8051,"thread_start_us":7894,"threads_started":1}
I20260807 18:15:00.398788 28430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.380974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:34151 (local address 127.0.0.1:46948)
0807 18:15:00.381748 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:00.381771 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:00.381905 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:00.390972 (+ 9067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:00.390987 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:00.391080 (+ 93us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:15:00.391762 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:00.391783 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:00.393864 (+ 2081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:00.393937 (+ 73us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:00.396059 (+ 2122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:00.396076 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:00.397452 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:00.397458 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:00.397697 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:00.397820 (+ 123us) client_negotiation.cc:769] Sending connection context
0807 18:15:00.397940 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 18:15:00.398200 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"mutex_wait_us":139,"thread_start_us":1342,"threads_started":1}
I20260807 18:15:00.399632 27686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "94d47e0385a840749cd3676abfda32c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "822d5b469a444f5ea371b43ecd44bb62" is_pre_election: true
I20260807 18:15:00.400020 27686 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94d47e0385a840749cd3676abfda32c0 in term 1.
I20260807 18:15:00.400352 28432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.391803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:37317 (local address 127.0.0.1:47484)
0807 18:15:00.391944 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:00.391962 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:00.392138 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:00.393345 (+ 1207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:00.393352 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:00.393380 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:15:00.393617 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:00.393626 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:00.395703 (+ 2077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:00.395714 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:00.397554 (+ 1840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:00.397569 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:00.399424 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:00.399433 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:00.399644 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:00.399671 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 18:15:00.399798 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 18:15:00.399988 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 18:15:00.400382 28439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.391994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47484 (local address 127.17.110.61:37317)
0807 18:15:00.392997 (+ 1003us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:00.393003 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:00.393029 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:15:00.393096 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:00.393101 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:00.393172 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:00.393811 (+ 639us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:15:00.394113 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:00.395519 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:00.397713 (+ 2194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:00.399184 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:00.399313 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:00.399455 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:00.399849 (+ 394us) server_negotiation.cc:299] Negotiation successful
0807 18:15:00.399997 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":828,"thread_start_us":1928,"threads_started":1}
I20260807 18:15:00.400775 27380 leader_election.cc:302] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 822d5b469a444f5ea371b43ecd44bb62, 94d47e0385a840749cd3676abfda32c0; no voters: ce9e1ae9d6354f9d9b7020b5e73f2ef2
I20260807 18:15:00.401183 28428 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 18:15:00.401350 28428 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ce9e1ae9d6354f9d9b7020b5e73f2ef2)
I20260807 18:15:00.401255 27408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "822d5b469a444f5ea371b43ecd44bb62" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "94d47e0385a840749cd3676abfda32c0" is_pre_election: true
I20260807 18:15:00.401463 28428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:15:00.403898 28428 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: false } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: false } }
I20260807 18:15:00.404374 27408 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 822d5b469a444f5ea371b43ecd44bb62 in current term 2: Already voted for candidate 94d47e0385a840749cd3676abfda32c0 in this term.
I20260807 18:15:00.404793 28428 leader_election.cc:288] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [CANDIDATE]: Term 2 election: Requested vote from peers ce9e1ae9d6354f9d9b7020b5e73f2ef2 (127.17.110.62:40597), 822d5b469a444f5ea371b43ecd44bb62 (127.17.110.60:34151)
I20260807 18:15:00.405514 28430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.404909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.0.0.1:58794)
0807 18:15:00.405203 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:00.405308 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
I20260807 18:15:00.405421 27686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "94d47e0385a840749cd3676abfda32c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "822d5b469a444f5ea371b43ecd44bb62"
I20260807 18:15:00.405755 27686 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:15:00.405928 27649 leader_election.cc:302] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 822d5b469a444f5ea371b43ecd44bb62; no voters: 94d47e0385a840749cd3676abfda32c0, ce9e1ae9d6354f9d9b7020b5e73f2ef2
I20260807 18:15:00.406735 28430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.406228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.0.0.1:58810)
0807 18:15:00.406489 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:00.406570 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
W20260807 18:15:00.406949 27377 leader_election.cc:334] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ce9e1ae9d6354f9d9b7020b5e73f2ef2 (127.17.110.62:40597): Network error: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
I20260807 18:15:00.408360 27686 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94d47e0385a840749cd3676abfda32c0 in term 2.
I20260807 18:15:00.408566 28427 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:15:00.408841 27380 leader_election.cc:302] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 822d5b469a444f5ea371b43ecd44bb62, 94d47e0385a840749cd3676abfda32c0; no voters: ce9e1ae9d6354f9d9b7020b5e73f2ef2
I20260807 18:15:00.409168 28428 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 18:15:00.409694 28428 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [term 2 LEADER]: Becoming Leader. State: Replica: 94d47e0385a840749cd3676abfda32c0, State: Running, Role: LEADER
I20260807 18:15:00.410127 28428 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: false } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: false } }
I20260807 18:15:00.412146 28443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94d47e0385a840749cd3676abfda32c0. Latest consensus state: current_term: 2 leader_uuid: "94d47e0385a840749cd3676abfda32c0" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: false } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: false } } }
I20260807 18:15:00.412514 28443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [sys.catalog]: This master's current role is: LEADER
I20260807 18:15:00.413228 28445 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:15:00.417575 28445 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:15:00.418555 28445 catalog_manager.cc:1259] Loaded cluster ID: 2749ae813a664a0f81bd97be8eb21721
I20260807 18:15:00.418678 28445 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:15:00.420492 28445 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:15:00.421412 28445 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0: Loaded TSK: 0
I20260807 18:15:00.422408 28445 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:15:00.478395 27394 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47474:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:15:00.486119 27394 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:15:00.491077 28430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.490553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.0.0.1:58834)
0807 18:15:00.490763 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:00.490902 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 18:15:00.491385 27686 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [term 2 FOLLOWER]: Refusing update from remote peer 94d47e0385a840749cd3676abfda32c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260807 18:15:00.492386 28430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.491855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.0.0.1:58838)
0807 18:15:00.492054 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:00.492168 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 18:15:00.492213 28443 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260807 18:15:00.492591 27377 consensus_peers.cc:487] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 -> Peer ce9e1ae9d6354f9d9b7020b5e73f2ef2 (127.17.110.62:40597): Couldn't send request to peer ce9e1ae9d6354f9d9b7020b5e73f2ef2. Status: Network error: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 18:15:00.495141 28427 sys_catalog.cc:455] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94d47e0385a840749cd3676abfda32c0. Latest consensus state: current_term: 2 leader_uuid: "94d47e0385a840749cd3676abfda32c0" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: false } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: false } } }
I20260807 18:15:00.495441 28427 sys_catalog.cc:458] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:15:00.497862 28428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "94d47e0385a840749cd3676abfda32c0" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: false } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: false } } }
I20260807 18:15:00.498376 28428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0 [sys.catalog]: This master's current role is: LEADER
I20260807 18:15:00.498876 28448 mvcc.cc:205] Tried to move back new op lower bound from 7315974146003513344 to 7315974145683714048. Current Snapshot: MvccSnapshot[applied={T|T < 7315974146003513344}]
I20260807 18:15:00.500212 28427 sys_catalog.cc:455] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "94d47e0385a840749cd3676abfda32c0" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ce9e1ae9d6354f9d9b7020b5e73f2ef2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40597 } } peers { permanent_uuid: "94d47e0385a840749cd3676abfda32c0" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 37317 } attrs { promote: false } } peers { permanent_uuid: "822d5b469a444f5ea371b43ecd44bb62" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34151 } attrs { promote: false } } }
I20260807 18:15:00.500531 28427 sys_catalog.cc:458] T 00000000000000000000000000000000 P 822d5b469a444f5ea371b43ecd44bb62 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:15:00.517016 28455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.508007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56384 (local address 127.17.110.1:33421)
0807 18:15:00.508517 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:00.508524 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:00.508544 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:15:00.508622 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:00.508630 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:00.508716 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:00.508838 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:15:00.511381 (+ 2543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:00.512481 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:00.514990 (+ 2509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:00.515943 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:00.516094 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:00.516209 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:00.516545 (+ 336us) server_negotiation.cc:299] Negotiation successful
0807 18:15:00.516715 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":259,"threads_started":1}
I20260807 18:15:00.517031 28430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.507931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:33421 (local address 127.0.0.1:56384)
0807 18:15:00.508177 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:00.508206 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:00.508355 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:00.508891 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:00.508896 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:00.508935 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:15:00.511138 (+ 2203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:00.511157 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:00.512670 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:00.512676 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:00.514304 (+ 1628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:00.514317 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:00.516140 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:00.516146 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:00.516351 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:00.516388 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 18:15:00.516495 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 18:15:00.516670 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":48}
I20260807 18:15:00.517808 28457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.509199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58642 (local address 127.17.110.2:40847)
0807 18:15:00.510317 (+ 1118us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:00.510324 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:00.510343 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:15:00.510415 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:00.510418 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:00.510474 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:00.510558 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:15:00.511485 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:00.512582 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:00.515385 (+ 2803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:00.516560 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:00.516684 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:00.516791 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:00.517360 (+ 569us) server_negotiation.cc:299] Negotiation successful
0807 18:15:00.517504 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1001,"thread_start_us":908,"threads_started":1}
I20260807 18:15:00.517833 28456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.508952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:40847 (local address 127.0.0.1:58642)
0807 18:15:00.509816 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:00.509836 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:00.509946 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:00.510664 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:00.510673 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:00.510764 (+ 91us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:15:00.511288 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:00.511299 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:00.512761 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:00.512766 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:00.515164 (+ 2398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:00.515181 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:00.516963 (+ 1782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:00.516970 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:00.517170 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:00.517205 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 18:15:00.517328 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 18:15:00.517462 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":694,"mutex_wait_us":139,"thread_start_us":665,"threads_started":1}
I20260807 18:15:00.518029 27836 tablet_service.cc:1467] Processing CreateTablet for tablet e7bfbd5bfaf9497b9840ab662073c5a6 (DEFAULT_TABLE table=test-table [id=aeee8d3e72054be0be01ba2a4ce766aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:15:00.518543 27836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7bfbd5bfaf9497b9840ab662073c5a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:00.519577 28431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.508947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:37809 (local address 127.0.0.1:50280)
0807 18:15:00.509103 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:00.509123 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:00.509267 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:00.511727 (+ 2460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:00.511734 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:00.511763 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:15:00.512080 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:00.512092 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:00.513503 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:00.513509 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:00.515712 (+ 2203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:00.515728 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:00.518824 (+ 3096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:00.518831 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:00.519007 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:00.519059 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 18:15:00.519149 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 18:15:00.519302 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 18:15:00.520460 28458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.509564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50280 (local address 127.17.110.3:37809)
0807 18:15:00.511301 (+ 1737us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:00.511308 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:00.511328 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:15:00.511397 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:00.511402 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:00.511524 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:00.511685 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:15:00.512263 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:00.513357 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:00.515899 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:00.517097 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:00.519818 (+ 2721us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:00.519940 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:00.519997 (+ 57us) server_negotiation.cc:299] Negotiation successful
0807 18:15:00.520155 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1594,"thread_start_us":1465,"threads_started":1}
I20260807 18:15:00.521212 28232 tablet_service.cc:1467] Processing CreateTablet for tablet e7bfbd5bfaf9497b9840ab662073c5a6 (DEFAULT_TABLE table=test-table [id=aeee8d3e72054be0be01ba2a4ce766aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:15:00.521627 28232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7bfbd5bfaf9497b9840ab662073c5a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:00.523573 28043 tablet_service.cc:1467] Processing CreateTablet for tablet e7bfbd5bfaf9497b9840ab662073c5a6 (DEFAULT_TABLE table=test-table [id=aeee8d3e72054be0be01ba2a4ce766aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:15:00.524062 28043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7bfbd5bfaf9497b9840ab662073c5a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:00.525002 28459 tablet_bootstrap.cc:492] T e7bfbd5bfaf9497b9840ab662073c5a6 P d6fda665629f44b69b25453b35fb9490: Bootstrap starting.
I20260807 18:15:00.526605 28459 tablet_bootstrap.cc:654] T e7bfbd5bfaf9497b9840ab662073c5a6 P d6fda665629f44b69b25453b35fb9490: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:00.529489 28459 tablet_bootstrap.cc:492] T e7bfbd5bfaf9497b9840ab662073c5a6 P d6fda665629f44b69b25453b35fb9490: No bootstrap required, opened a new log
I20260807 18:15:00.529666 28459 ts_tablet_manager.cc:1397] T e7bfbd5bfaf9497b9840ab662073c5a6 P d6fda665629f44b69b25453b35fb9490: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 18:15:00.530378 28459 raft_consensus.cc:348] T e7bfbd5bfaf9497b9840ab662073c5a6 P d6fda665629f44b69b25453b35fb9490 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6fda665629f44b69b25453b35fb9490" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 33421 } } peers { permanent_uuid: "480a54d1b82d42acb4428942491d0910" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40847 } } peers { permanent_uuid: "25c40cc9663e47c8b30a95e33060db88" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37809 } }
I20260807 18:15:00.530606 28459 raft_consensus.cc:374] T e7bfbd5bfaf9497b9840ab662073c5a6 P d6fda665629f44b69b25453b35fb9490 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:00.530705 28459 raft_consensus.cc:729] T e7bfbd5bfaf9497b9840ab662073c5a6 P d6fda665629f44b69b25453b35fb9490 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6fda665629f44b69b25453b35fb9490, State: Initialized, Role: FOLLOWER
I20260807 18:15:00.531106 28459 consensus_queue.cc:260] T e7bfbd5bfaf9497b9840ab662073c5a6 P d6fda665629f44b69b25453b35fb9490 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6fda665629f44b69b25453b35fb9490" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 33421 } } peers { permanent_uuid: "480a54d1b82d42acb4428942491d0910" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40847 } } peers { permanent_uuid: "25c40cc9663e47c8b30a95e33060db88" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37809 } }
I20260807 18:15:00.532063 28459 ts_tablet_manager.cc:1428] T e7bfbd5bfaf9497b9840ab662073c5a6 P d6fda665629f44b69b25453b35fb9490: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 18:15:00.539539 28461 tablet_bootstrap.cc:492] T e7bfbd5bfaf9497b9840ab662073c5a6 P 25c40cc9663e47c8b30a95e33060db88: Bootstrap starting.
I20260807 18:15:00.542361 28461 tablet_bootstrap.cc:654] T e7bfbd5bfaf9497b9840ab662073c5a6 P 25c40cc9663e47c8b30a95e33060db88: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:00.543905 28466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.532662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.17.110.1:44129)
0807 18:15:00.543298 (+ 10636us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:00.543578 (+ 280us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10429,"thread_start_us":10244,"threads_started":1}
I20260807 18:15:00.544699 28465 tablet_bootstrap.cc:492] T e7bfbd5bfaf9497b9840ab662073c5a6 P 480a54d1b82d42acb4428942491d0910: Bootstrap starting.
I20260807 18:15:00.546559 28466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.546095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.17.110.1:58667)
0807 18:15:00.546287 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:00.546372 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 18:15:00.546691 27932 heartbeater.cc:503] Master 127.17.110.61:37317 was elected leader, sending a full tablet report...
I20260807 18:15:00.546967 28461 tablet_bootstrap.cc:492] T e7bfbd5bfaf9497b9840ab662073c5a6 P 25c40cc9663e47c8b30a95e33060db88: No bootstrap required, opened a new log
I20260807 18:15:00.547168 28461 ts_tablet_manager.cc:1397] T e7bfbd5bfaf9497b9840ab662073c5a6 P 25c40cc9663e47c8b30a95e33060db88: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260807 18:15:00.547465 28465 tablet_bootstrap.cc:654] T e7bfbd5bfaf9497b9840ab662073c5a6 P 480a54d1b82d42acb4428942491d0910: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:00.548144 28461 raft_consensus.cc:348] T e7bfbd5bfaf9497b9840ab662073c5a6 P 25c40cc9663e47c8b30a95e33060db88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6fda665629f44b69b25453b35fb9490" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 33421 } } peers { permanent_uuid: "480a54d1b82d42acb4428942491d0910" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40847 } } peers { permanent_uuid: "25c40cc9663e47c8b30a95e33060db88" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37809 } }
I20260807 18:15:00.548399 28461 raft_consensus.cc:374] T e7bfbd5bfaf9497b9840ab662073c5a6 P 25c40cc9663e47c8b30a95e33060db88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:00.548491 28461 raft_consensus.cc:729] T e7bfbd5bfaf9497b9840ab662073c5a6 P 25c40cc9663e47c8b30a95e33060db88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25c40cc9663e47c8b30a95e33060db88, State: Initialized, Role: FOLLOWER
I20260807 18:15:00.548796 28461 consensus_queue.cc:260] T e7bfbd5bfaf9497b9840ab662073c5a6 P 25c40cc9663e47c8b30a95e33060db88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6fda665629f44b69b25453b35fb9490" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 33421 } } peers { permanent_uuid: "480a54d1b82d42acb4428942491d0910" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40847 } } peers { permanent_uuid: "25c40cc9663e47c8b30a95e33060db88" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37809 } }
I20260807 18:15:00.549640 28461 ts_tablet_manager.cc:1428] T e7bfbd5bfaf9497b9840ab662073c5a6 P 25c40cc9663e47c8b30a95e33060db88: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260807 18:15:00.550053 28465 tablet_bootstrap.cc:492] T e7bfbd5bfaf9497b9840ab662073c5a6 P 480a54d1b82d42acb4428942491d0910: No bootstrap required, opened a new log
I20260807 18:15:00.550298 28465 ts_tablet_manager.cc:1397] T e7bfbd5bfaf9497b9840ab662073c5a6 P 480a54d1b82d42acb4428942491d0910: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 18:15:00.551532 28471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.550029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.17.110.3:46717)
0807 18:15:00.551091 (+ 1062us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:00.551265 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":909,"thread_start_us":690,"threads_started":1}
I20260807 18:15:00.551381 28465 raft_consensus.cc:348] T e7bfbd5bfaf9497b9840ab662073c5a6 P 480a54d1b82d42acb4428942491d0910 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6fda665629f44b69b25453b35fb9490" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 33421 } } peers { permanent_uuid: "480a54d1b82d42acb4428942491d0910" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40847 } } peers { permanent_uuid: "25c40cc9663e47c8b30a95e33060db88" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37809 } }
I20260807 18:15:00.551640 28465 raft_consensus.cc:374] T e7bfbd5bfaf9497b9840ab662073c5a6 P 480a54d1b82d42acb4428942491d0910 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:00.551782 28465 raft_consensus.cc:729] T e7bfbd5bfaf9497b9840ab662073c5a6 P 480a54d1b82d42acb4428942491d0910 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 480a54d1b82d42acb4428942491d0910, State: Initialized, Role: FOLLOWER
I20260807 18:15:00.552114 28465 consensus_queue.cc:260] T e7bfbd5bfaf9497b9840ab662073c5a6 P 480a54d1b82d42acb4428942491d0910 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6fda665629f44b69b25453b35fb9490" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 33421 } } peers { permanent_uuid: "480a54d1b82d42acb4428942491d0910" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40847 } } peers { permanent_uuid: "25c40cc9663e47c8b30a95e33060db88" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37809 } }
I20260807 18:15:00.555258 28323 heartbeater.cc:503] Master 127.17.110.61:37317 was elected leader, sending a full tablet report...
I20260807 18:15:00.555605 28472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.552198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.17.110.3:40231)
0807 18:15:00.554950 (+ 2752us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:00.555193 (+ 243us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2605,"thread_start_us":2449,"threads_started":1}
I20260807 18:15:00.555733 28465 ts_tablet_manager.cc:1428] T e7bfbd5bfaf9497b9840ab662073c5a6 P 480a54d1b82d42acb4428942491d0910: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260807 18:15:00.557907 28475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.556352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.17.110.2:59547)
0807 18:15:00.557083 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:00.557599 (+ 516us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":409,"threads_started":1}
I20260807 18:15:00.814741 28470 raft_consensus.cc:482] T e7bfbd5bfaf9497b9840ab662073c5a6 P 25c40cc9663e47c8b30a95e33060db88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:15:00.815032 28470 raft_consensus.cc:504] T e7bfbd5bfaf9497b9840ab662073c5a6 P 25c40cc9663e47c8b30a95e33060db88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6fda665629f44b69b25453b35fb9490" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 33421 } } peers { permanent_uuid: "480a54d1b82d42acb4428942491d0910" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40847 } } peers { permanent_uuid: "25c40cc9663e47c8b30a95e33060db88" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37809 } }
I20260807 18:15:00.816677 28470 leader_election.cc:288] T e7bfbd5bfaf9497b9840ab662073c5a6 P 25c40cc9663e47c8b30a95e33060db88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6fda665629f44b69b25453b35fb9490 (127.17.110.1:33421), 480a54d1b82d42acb4428942491d0910 (127.17.110.2:40847)
I20260807 18:15:00.822955 28457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.817451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:39385 (local address 127.17.110.2:40847)
0807 18:15:00.817652 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:00.817657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:00.817672 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:15:00.817733 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:00.817737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:00.817792 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:00.817901 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:15:00.818432 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:00.819282 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:00.821107 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:00.821911 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:00.822105 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:00.822190 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:00.822502 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 18:15:00.822647 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 18:15:00.822999 28472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.816992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:40847 (local address 127.17.110.3:39385)
0807 18:15:00.817169 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:00.817191 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:00.817351 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:00.817926 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:00.817930 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:00.817979 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:15:00.818276 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:00.818285 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:00.819444 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:00.819450 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:00.820920 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:00.820934 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:00.822119 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:00.822126 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:00.822333 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:00.822359 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:15:00.822473 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 18:15:00.822631 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"spinlock_wait_cycles":1792}
I20260807 18:15:00.823014 28455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.817069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:34797 (local address 127.17.110.1:33421)
0807 18:15:00.817274 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:00.817282 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:00.817502 (+ 220us) server_negotiation.cc:407] Connection header received
0807 18:15:00.817715 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:00.817721 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:00.817792 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:00.817894 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:15:00.818393 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:00.819347 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:00.821219 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:00.822015 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:00.822217 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:00.822295 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:00.822582 (+ 287us) server_negotiation.cc:299] Negotiation successful
0807 18:15:00.822748 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 18:15:00.823061 28471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.816997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:33421 (local address 127.17.110.3:34797)
0807 18:15:00.817382 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:00.817400 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:00.817566 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:00.817918 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:00.817923 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:00.817963 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:15:00.818242 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:00.818254 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:00.819580 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:00.819586 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:00.821022 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:00.821035 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:00.822221 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:00.822231 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:00.822414 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:00.822438 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 18:15:00.822550 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 18:15:00.822730 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":45,"spinlock_wait_cycles":2944}
I20260807 18:15:00.823866 28068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7bfbd5bfaf9497b9840ab662073c5a6" candidate_uuid: "25c40cc9663e47c8b30a95e33060db88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "480a54d1b82d42acb4428942491d0910" is_pre_election: true
I20260807 18:15:00.824159 28068 raft_consensus.cc:2454] T e7bfbd5bfaf9497b9840ab662073c5a6 P 480a54d1b82d42acb4428942491d0910 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25c40cc9663e47c8b30a95e33060db88 in term 0.
I20260807 18:15:00.824319 27862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7bfbd5bfaf9497b9840ab662073c5a6" candidate_uuid: "25c40cc9663e47c8b30a95e33060db88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6fda665629f44b69b25453b35fb9490" is_pre_election: true
I20260807 18:15:00.824796 27862 raft_consensus.cc:2454] T e7bfbd5bfaf9497b9840ab662073c5a6 P d6fda665629f44b69b25453b35fb9490 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25c40cc9663e47c8b30a95e33060db88 in term 0.
I20260807 18:15:00.824771 28179 leader_election.cc:302] T e7bfbd5bfaf9497b9840ab662073c5a6 P 25c40cc9663e47c8b30a95e33060db88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25c40cc9663e47c8b30a95e33060db88, 480a54d1b82d42acb4428942491d0910; no voters:
I20260807 18:15:00.825238 28470 raft_consensus.cc:2781] T e7bfbd5bfaf9497b9840ab662073c5a6 P 25c40cc9663e47c8b30a95e33060db88 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:15:00.825383 28470 raft_consensus.cc:482] T e7bfbd5bfaf9497b9840ab662073c5a6 P 25c40cc9663e47c8b30a95e33060db88 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:15:00.825489 28470 raft_consensus.cc:3037] T e7bfbd5bfaf9497b9840ab662073c5a6 P 25c40cc9663e47c8b30a95e33060db88 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:00.827808 28470 raft_consensus.cc:504] T e7bfbd5bfaf9497b9840ab662073c5a6 P 25c40cc9663e47c8b30a95e33060db88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6fda665629f44b69b25453b35fb9490" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 33421 } } peers { permanent_uuid: "480a54d1b82d42acb4428942491d0910" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40847 } } peers { permanent_uuid: "25c40cc9663e47c8b30a95e33060db88" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37809 } }
I20260807 18:15:00.828464 28470 leader_election.cc:288] T e7bfbd5bfaf9497b9840ab662073c5a6 P 25c40cc9663e47c8b30a95e33060db88 [CANDIDATE]: Term 1 election: Requested vote from peers d6fda665629f44b69b25453b35fb9490 (127.17.110.1:33421), 480a54d1b82d42acb4428942491d0910 (127.17.110.2:40847)
I20260807 18:15:00.829075 27862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7bfbd5bfaf9497b9840ab662073c5a6" candidate_uuid: "25c40cc9663e47c8b30a95e33060db88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6fda665629f44b69b25453b35fb9490"
I20260807 18:15:00.829080 28068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7bfbd5bfaf9497b9840ab662073c5a6" candidate_uuid: "25c40cc9663e47c8b30a95e33060db88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "480a54d1b82d42acb4428942491d0910"
I20260807 18:15:00.829298 28068 raft_consensus.cc:3037] T e7bfbd5bfaf9497b9840ab662073c5a6 P 480a54d1b82d42acb4428942491d0910 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:00.829288 27862 raft_consensus.cc:3037] T e7bfbd5bfaf9497b9840ab662073c5a6 P d6fda665629f44b69b25453b35fb9490 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:00.831470 28068 raft_consensus.cc:2454] T e7bfbd5bfaf9497b9840ab662073c5a6 P 480a54d1b82d42acb4428942491d0910 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25c40cc9663e47c8b30a95e33060db88 in term 1.
I20260807 18:15:00.831585 27862 raft_consensus.cc:2454] T e7bfbd5bfaf9497b9840ab662073c5a6 P d6fda665629f44b69b25453b35fb9490 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25c40cc9663e47c8b30a95e33060db88 in term 1.
I20260807 18:15:00.832198 28180 leader_election.cc:302] T e7bfbd5bfaf9497b9840ab662073c5a6 P 25c40cc9663e47c8b30a95e33060db88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25c40cc9663e47c8b30a95e33060db88, d6fda665629f44b69b25453b35fb9490; no voters:
I20260807 18:15:00.832556 28470 raft_consensus.cc:2781] T e7bfbd5bfaf9497b9840ab662073c5a6 P 25c40cc9663e47c8b30a95e33060db88 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:15:00.833030 28470 raft_consensus.cc:686] T e7bfbd5bfaf9497b9840ab662073c5a6 P 25c40cc9663e47c8b30a95e33060db88 [term 1 LEADER]: Becoming Leader. State: Replica: 25c40cc9663e47c8b30a95e33060db88, State: Running, Role: LEADER
I20260807 18:15:00.833484 28470 consensus_queue.cc:237] T e7bfbd5bfaf9497b9840ab662073c5a6 P 25c40cc9663e47c8b30a95e33060db88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6fda665629f44b69b25453b35fb9490" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 33421 } } peers { permanent_uuid: "480a54d1b82d42acb4428942491d0910" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40847 } } peers { permanent_uuid: "25c40cc9663e47c8b30a95e33060db88" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37809 } }
I20260807 18:15:00.834142 28471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.833703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.17.110.3:47147)
0807 18:15:00.833886 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:00.833978 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 18:15:00.837878 27393 catalog_manager.cc:5495] T e7bfbd5bfaf9497b9840ab662073c5a6 P 25c40cc9663e47c8b30a95e33060db88 reported cstate change: term changed from 0 to 1, leader changed from <none> to 25c40cc9663e47c8b30a95e33060db88 (127.17.110.3). New cstate: current_term: 1 leader_uuid: "25c40cc9663e47c8b30a95e33060db88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6fda665629f44b69b25453b35fb9490" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 33421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "480a54d1b82d42acb4428942491d0910" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25c40cc9663e47c8b30a95e33060db88" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 37809 } health_report { overall_health: HEALTHY } } }
I20260807 18:15:00.895582 28458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.890089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50292 (local address 127.17.110.3:37809)
0807 18:15:00.890384 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:00.890392 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:00.890415 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:15:00.890533 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:00.890537 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:00.890629 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:00.890861 (+ 232us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:15:00.891281 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:00.892062 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:00.893357 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:00.893785 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:00.893930 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:00.894363 (+ 433us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:15:00.894995 (+ 632us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:15:00.895164 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:00.895220 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 18:15:00.895307 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 18:15:00.900260 28068 raft_consensus.cc:1261] T e7bfbd5bfaf9497b9840ab662073c5a6 P 480a54d1b82d42acb4428942491d0910 [term 1 FOLLOWER]: Refusing update from remote peer 25c40cc9663e47c8b30a95e33060db88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:15:00.900718 27862 raft_consensus.cc:1261] T e7bfbd5bfaf9497b9840ab662073c5a6 P d6fda665629f44b69b25453b35fb9490 [term 1 FOLLOWER]: Refusing update from remote peer 25c40cc9663e47c8b30a95e33060db88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:15:00.901454 28476 consensus_queue.cc:1035] T e7bfbd5bfaf9497b9840ab662073c5a6 P 25c40cc9663e47c8b30a95e33060db88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "480a54d1b82d42acb4428942491d0910" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:15:00.901887 28470 consensus_queue.cc:1035] T e7bfbd5bfaf9497b9840ab662073c5a6 P 25c40cc9663e47c8b30a95e33060db88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6fda665629f44b69b25453b35fb9490" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 33421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:15:00.903589 28126 heartbeater.cc:503] Master 127.17.110.61:37317 was elected leader, sending a full tablet report...
I20260807 18:15:00.904901 28475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.903624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.17.110.2:35503)
0807 18:15:00.904632 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:00.904743 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":888}
I20260807 18:15:00.905915 28466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.905226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.17.110.1:50305)
0807 18:15:00.905511 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:00.905685 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260807 18:15:00.916363 28466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.914552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.17.110.1:40111)
0807 18:15:00.916074 (+ 1522us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:00.916185 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1392}
I20260807 18:15:00.917526 28471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.916741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.17.110.3:43497)
0807 18:15:00.917244 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:00.917357 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375}
I20260807 18:15:00.921294 28471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.920680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.17.110.3:40059)
0807 18:15:00.920967 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:00.921065 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
I20260807 18:15:00.924316 28475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.923658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.17.110.2:53249)
0807 18:15:00.923958 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:00.924088 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
I20260807 18:15:00.931509 27394 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:15:00.931813 27394 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:15:00.938423 27394 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 94d47e0385a840749cd3676abfda32c0: Sending DeleteTablet for 3 replicas of tablet e7bfbd5bfaf9497b9840ab662073c5a6
I20260807 18:15:00.939739 27836 tablet_service.cc:1514] Processing DeleteTablet for tablet e7bfbd5bfaf9497b9840ab662073c5a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:15:00 UTC) from {username='oryx', principal='oryx/127.17.110.61@KRBTEST.COM'} at 127.0.0.1:56384
I20260807 18:15:00.940176 28232 tablet_service.cc:1514] Processing DeleteTablet for tablet e7bfbd5bfaf9497b9840ab662073c5a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:15:00 UTC) from {username='oryx', principal='oryx/127.17.110.61@KRBTEST.COM'} at 127.0.0.1:50280
I20260807 18:15:00.940335 28043 tablet_service.cc:1514] Processing DeleteTablet for tablet e7bfbd5bfaf9497b9840ab662073c5a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:15:00 UTC) from {username='oryx', principal='oryx/127.17.110.61@KRBTEST.COM'} at 127.0.0.1:58642
I20260807 18:15:00.943473 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 27631
I20260807 18:15:00.952404 28493 tablet_replica.cc:330] T e7bfbd5bfaf9497b9840ab662073c5a6 P d6fda665629f44b69b25453b35fb9490: stopping tablet replica
I20260807 18:15:00.952680 28493 raft_consensus.cc:2229] T e7bfbd5bfaf9497b9840ab662073c5a6 P d6fda665629f44b69b25453b35fb9490 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:15:00.953064 28493 raft_consensus.cc:2258] T e7bfbd5bfaf9497b9840ab662073c5a6 P d6fda665629f44b69b25453b35fb9490 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:15:00.953236 28495 tablet_replica.cc:330] T e7bfbd5bfaf9497b9840ab662073c5a6 P 25c40cc9663e47c8b30a95e33060db88: stopping tablet replica
I20260807 18:15:00.953477 28495 raft_consensus.cc:2229] T e7bfbd5bfaf9497b9840ab662073c5a6 P 25c40cc9663e47c8b30a95e33060db88 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:15:00.953676 28496 tablet_replica.cc:330] T e7bfbd5bfaf9497b9840ab662073c5a6 P 480a54d1b82d42acb4428942491d0910: stopping tablet replica
I20260807 18:15:00.954248 28495 raft_consensus.cc:2258] T e7bfbd5bfaf9497b9840ab662073c5a6 P 25c40cc9663e47c8b30a95e33060db88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:15:00.955387 28493 ts_tablet_manager.cc:1905] T e7bfbd5bfaf9497b9840ab662073c5a6 P d6fda665629f44b69b25453b35fb9490: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:15:00.955641 28495 ts_tablet_manager.cc:1905] T e7bfbd5bfaf9497b9840ab662073c5a6 P 25c40cc9663e47c8b30a95e33060db88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:15:00.955638 28496 raft_consensus.cc:2229] T e7bfbd5bfaf9497b9840ab662073c5a6 P 480a54d1b82d42acb4428942491d0910 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:15:00.957197 28496 raft_consensus.cc:2258] T e7bfbd5bfaf9497b9840ab662073c5a6 P 480a54d1b82d42acb4428942491d0910 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:15:00.959934 28496 ts_tablet_manager.cc:1905] T e7bfbd5bfaf9497b9840ab662073c5a6 P 480a54d1b82d42acb4428942491d0910: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:15:00.963063 28493 ts_tablet_manager.cc:1918] T e7bfbd5bfaf9497b9840ab662073c5a6 P d6fda665629f44b69b25453b35fb9490: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:15:00.963243 28493 log.cc:1196] T e7bfbd5bfaf9497b9840ab662073c5a6 P d6fda665629f44b69b25453b35fb9490: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-0/wal/wals/e7bfbd5bfaf9497b9840ab662073c5a6
I20260807 18:15:00.963752 28493 ts_tablet_manager.cc:1939] T e7bfbd5bfaf9497b9840ab662073c5a6 P d6fda665629f44b69b25453b35fb9490: Deleting consensus metadata
I20260807 18:15:00.965390 27381 catalog_manager.cc:4841] TS d6fda665629f44b69b25453b35fb9490 (127.17.110.1:33421): tablet e7bfbd5bfaf9497b9840ab662073c5a6 (table test-table [id=aeee8d3e72054be0be01ba2a4ce766aa]) successfully deleted
I20260807 18:15:00.967180 28496 ts_tablet_manager.cc:1918] T e7bfbd5bfaf9497b9840ab662073c5a6 P 480a54d1b82d42acb4428942491d0910: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:15:00.967392 28496 log.cc:1196] T e7bfbd5bfaf9497b9840ab662073c5a6 P 480a54d1b82d42acb4428942491d0910: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-1/wal/wals/e7bfbd5bfaf9497b9840ab662073c5a6
I20260807 18:15:00.967917 28496 ts_tablet_manager.cc:1939] T e7bfbd5bfaf9497b9840ab662073c5a6 P 480a54d1b82d42acb4428942491d0910: Deleting consensus metadata
I20260807 18:15:00.969308 27380 catalog_manager.cc:4841] TS 480a54d1b82d42acb4428942491d0910 (127.17.110.2:40847): tablet e7bfbd5bfaf9497b9840ab662073c5a6 (table test-table [id=aeee8d3e72054be0be01ba2a4ce766aa]) successfully deleted
I20260807 18:15:00.971500 28495 ts_tablet_manager.cc:1918] T e7bfbd5bfaf9497b9840ab662073c5a6 P 25c40cc9663e47c8b30a95e33060db88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:15:00.971704 28495 log.cc:1196] T e7bfbd5bfaf9497b9840ab662073c5a6 P 25c40cc9663e47c8b30a95e33060db88: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126429568977-17848-0/minicluster-data/ts-2/wal/wals/e7bfbd5bfaf9497b9840ab662073c5a6
I20260807 18:15:00.972294 28495 ts_tablet_manager.cc:1939] T e7bfbd5bfaf9497b9840ab662073c5a6 P 25c40cc9663e47c8b30a95e33060db88: Deleting consensus metadata
I20260807 18:15:00.973600 27377 catalog_manager.cc:4841] TS 25c40cc9663e47c8b30a95e33060db88 (127.17.110.3:37809): tablet e7bfbd5bfaf9497b9840ab662073c5a6 (table test-table [id=aeee8d3e72054be0be01ba2a4ce766aa]) successfully deleted
I20260807 18:15:00.974009 28455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:00.945409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56394 (local address 127.17.110.1:33421)
0807 18:15:00.945679 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:00.945689 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:00.953838 (+ 8149us) server_negotiation.cc:407] Connection header received
0807 18:15:00.954083 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:00.954091 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:00.954201 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:00.954404 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:00.957573 (+ 3169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:00.958760 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:00.960807 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:00.961575 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:00.961717 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:00.966071 (+ 4354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:00.966089 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:00.966105 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:00.966146 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:00.970224 (+ 4078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:00.971341 (+ 1117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:00.971353 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:00.971356 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:00.971546 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:00.972108 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:00.972115 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:00.972120 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:00.972662 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:00.973010 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:00.973344 (+ 334us) server_negotiation.cc:299] Negotiation successful
0807 18:15:00.973601 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 18:15:00.975073 27921 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:01.057170 28501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:01.055922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.0.0.1:58852)
0807 18:15:01.056874 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:01.056979 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":814,"thread_start_us":683,"threads_started":1}
I20260807 18:15:01.058874 28501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:01.058374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.0.0.1:58860)
0807 18:15:01.058552 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:01.058658 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 18:15:01.155953 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 27631
I20260807 18:15:01.188053 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 27947
I20260807 18:15:01.204052 28457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:01.188936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58650 (local address 127.17.110.2:40847)
0807 18:15:01.189115 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:01.189123 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:01.189283 (+ 160us) server_negotiation.cc:407] Connection header received
0807 18:15:01.189464 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:01.189470 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:01.189604 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:01.189750 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:01.191320 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:01.192166 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:01.193451 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:01.193878 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:01.194027 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:01.196812 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:01.196840 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:01.196855 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:01.196900 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:01.200346 (+ 3446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:01.201154 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:01.201167 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:01.201175 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:01.201333 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:01.201944 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:01.201952 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:01.201959 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:01.202417 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:01.202833 (+ 416us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:01.203387 (+ 554us) server_negotiation.cc:299] Negotiation successful
0807 18:15:01.203647 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 18:15:01.204880 28118 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:01.308053 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 27947
I20260807 18:15:01.334460 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 28149
I20260807 18:15:01.353758 28458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:01.335362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36268 (local address 127.17.110.3:37809)
0807 18:15:01.335595 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:01.335601 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:01.335627 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:15:01.335757 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:01.335768 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:01.335861 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:01.335985 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:01.337634 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:01.339054 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:01.340913 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:01.341666 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:01.341827 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:01.345395 (+ 3568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:01.345426 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:01.345450 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:01.345512 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:01.350141 (+ 4629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:01.351122 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:01.351132 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:01.351136 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:01.351354 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:01.351806 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:01.351812 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:01.351816 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:01.352325 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:01.352698 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:01.353056 (+ 358us) server_negotiation.cc:299] Negotiation successful
0807 18:15:01.353316 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260807 18:15:01.354657 28316 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:01.492105 28501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:01.491556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.0.0.1:48824)
0807 18:15:01.491782 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:01.491895 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 18:15:01.493307 28501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:01.492904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40597 (local address 127.0.0.1:48840)
0807 18:15:01.493064 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:01.493155 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:40597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 18:15:01.497781 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 28149
I20260807 18:15:01.533236 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 27204
I20260807 18:15:01.534509 27436 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:01.659251 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 27204
I20260807 18:15:01.689810 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 27342
I20260807 18:15:01.691274 27716 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:01.810284 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 27342
2026-08-07T18:15:01Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (11410 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:15:01 dist-test-slave-39l8 krb5kdc[28513](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:15:01 dist-test-slave-39l8 krb5kdc[28513](info): set up 2 sockets
Aug 07 18:15:01 dist-test-slave-39l8 krb5kdc[28513](info): commencing operation
krb5kdc: starting...
W20260807 18:15:03.449671 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.580s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:15:03 dist-test-slave-39l8 krb5kdc[28513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:15:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:15:03Z Disabled control of system clock
WARNING: no policy specified for oryx/127.17.110.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.110.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.110.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.62.keytab.
Entry for principal oryx/127.17.110.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.62.keytab.
I20260807 18:15:03.746122 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:40437
--webserver_interface=127.17.110.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.62.keytab
--principal=oryx/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:37387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:40437
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:15:04.144822 28529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:04.145195 28529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:04.145424 28529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:04.154670 28529 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:15:04.154914 28529 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 18:15:04.155015 28529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:04.155129 28529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:04.155231 28529 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:15:04.155349 28529 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:15:04.167645 28529 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:37387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:40437
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.62.keytab
--principal=oryx/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:40437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:15:04.169414 28529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:04.171463 28529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:04.187773 28538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:04.187861 28536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:04.187883 28535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:04.188377 28529 server_base.cc:1034] running on GCE node
I20260807 18:15:04.189401 28529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:04.191080 28529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:04.192353 28529 hybrid_clock.cc:648] HybridClock initialized: now 1786126504192289 us; error 37 us; skew 500 ppm
I20260807 18:15:04.192700 28529 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:15:04 dist-test-slave-39l8 krb5kdc[28513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126504, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:04.200760 28529 init.cc:377] Logged in from keytab as oryx/127.17.110.62@KRBTEST.COM (short username oryx)
I20260807 18:15:04.202931 28529 webserver.cc:460] Webserver started at http://127.17.110.62:40163/ using document root <none> and password file <none>
I20260807 18:15:04.203846 28529 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:04.204010 28529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:04.204407 28529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:15:04.207955 28529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "f89b8e74b0ae4f2daab1653a7fbb4e00"
format_stamp: "Formatted at 2026-08-07 18:15:04 on dist-test-slave-39l8"
server_key: "0003fb44b6e68b9ded53f2b6792bbeb7"
server_key_iv: "4c908b3128a88105d8772e6e293c5673"
server_key_version: "encryptionkey@0"
I20260807 18:15:04.208930 28529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "f89b8e74b0ae4f2daab1653a7fbb4e00"
format_stamp: "Formatted at 2026-08-07 18:15:04 on dist-test-slave-39l8"
server_key: "0003fb44b6e68b9ded53f2b6792bbeb7"
server_key_iv: "4c908b3128a88105d8772e6e293c5673"
server_key_version: "encryptionkey@0"
I20260807 18:15:04.215708 28529 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 18:15:04.220399 28549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:04.222925 28529 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:15:04.223232 28529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "f89b8e74b0ae4f2daab1653a7fbb4e00"
format_stamp: "Formatted at 2026-08-07 18:15:04 on dist-test-slave-39l8"
server_key: "0003fb44b6e68b9ded53f2b6792bbeb7"
server_key_iv: "4c908b3128a88105d8772e6e293c5673"
server_key_version: "encryptionkey@0"
I20260807 18:15:04.223541 28529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:04.252925 28529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:04.257509 28529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:04.257835 28529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:04.282809 28529 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:40437
I20260807 18:15:04.282868 28639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:40437 every 8 connection(s)
I20260807 18:15:04.284492 28529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:15:04.288200 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 28529
I20260807 18:15:04.288439 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:15:04.289317 17848 external_mini_cluster.cc:1428] Setting key 2a29d16e9ccca1b7c779d89c5301949d
I20260807 18:15:04.289755 28640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:04.306155 28640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f89b8e74b0ae4f2daab1653a7fbb4e00: Bootstrap starting.
Aug 07 18:15:04 dist-test-slave-39l8 krb5kdc[28513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM
I20260807 18:15:04.310877 28640 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f89b8e74b0ae4f2daab1653a7fbb4e00: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:04.312363 28640 log.cc:824] T 00000000000000000000000000000000 P f89b8e74b0ae4f2daab1653a7fbb4e00: Log is configured to *not* fsync() on all Append() calls
I20260807 18:15:04.315918 28640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f89b8e74b0ae4f2daab1653a7fbb4e00: No bootstrap required, opened a new log
I20260807 18:15:04.319064 28647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:04.291645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50764 (local address 127.17.110.62:40437)
0807 18:15:04.292558 (+ 913us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:04.292571 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:04.292614 (+ 43us) server_negotiation.cc:407] Connection header received
0807 18:15:04.295287 (+ 2673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:04.295318 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:04.295819 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:04.296302 (+ 483us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:04.298474 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:04.300782 (+ 2308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:04.303056 (+ 2274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:04.303888 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:04.304131 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:04.309562 (+ 5431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:04.309573 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:04.309591 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:04.309648 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:04.313809 (+ 4161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:04.314819 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:04.314828 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:04.314836 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:04.315057 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:04.315544 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:04.315549 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:04.315552 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:04.316269 (+ 717us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:04.316651 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:04.317196 (+ 545us) server_negotiation.cc:299] Negotiation successful
0807 18:15:04.317730 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":700,"thread_start_us":469,"threads_started":1}
I20260807 18:15:04.324548 28640 raft_consensus.cc:348] T 00000000000000000000000000000000 P f89b8e74b0ae4f2daab1653a7fbb4e00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89b8e74b0ae4f2daab1653a7fbb4e00" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40437 } }
I20260807 18:15:04.324949 28640 raft_consensus.cc:374] T 00000000000000000000000000000000 P f89b8e74b0ae4f2daab1653a7fbb4e00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:04.325084 28640 raft_consensus.cc:729] T 00000000000000000000000000000000 P f89b8e74b0ae4f2daab1653a7fbb4e00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f89b8e74b0ae4f2daab1653a7fbb4e00, State: Initialized, Role: FOLLOWER
I20260807 18:15:04.325829 28640 consensus_queue.cc:260] T 00000000000000000000000000000000 P f89b8e74b0ae4f2daab1653a7fbb4e00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89b8e74b0ae4f2daab1653a7fbb4e00" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40437 } }
I20260807 18:15:04.326054 28640 raft_consensus.cc:388] T 00000000000000000000000000000000 P f89b8e74b0ae4f2daab1653a7fbb4e00 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:15:04.326205 28640 raft_consensus.cc:482] T 00000000000000000000000000000000 P f89b8e74b0ae4f2daab1653a7fbb4e00 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:15:04.326406 28640 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f89b8e74b0ae4f2daab1653a7fbb4e00 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:04.329460 28640 raft_consensus.cc:504] T 00000000000000000000000000000000 P f89b8e74b0ae4f2daab1653a7fbb4e00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89b8e74b0ae4f2daab1653a7fbb4e00" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40437 } }
I20260807 18:15:04.330107 28640 leader_election.cc:302] T 00000000000000000000000000000000 P f89b8e74b0ae4f2daab1653a7fbb4e00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f89b8e74b0ae4f2daab1653a7fbb4e00; no voters:
I20260807 18:15:04.330902 28640 leader_election.cc:288] T 00000000000000000000000000000000 P f89b8e74b0ae4f2daab1653a7fbb4e00 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:15:04.331094 28649 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f89b8e74b0ae4f2daab1653a7fbb4e00 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:15:04.331874 28649 raft_consensus.cc:686] T 00000000000000000000000000000000 P f89b8e74b0ae4f2daab1653a7fbb4e00 [term 1 LEADER]: Becoming Leader. State: Replica: f89b8e74b0ae4f2daab1653a7fbb4e00, State: Running, Role: LEADER
I20260807 18:15:04.332535 28649 consensus_queue.cc:237] T 00000000000000000000000000000000 P f89b8e74b0ae4f2daab1653a7fbb4e00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89b8e74b0ae4f2daab1653a7fbb4e00" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40437 } }
I20260807 18:15:04.333344 28640 sys_catalog.cc:564] T 00000000000000000000000000000000 P f89b8e74b0ae4f2daab1653a7fbb4e00 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:15:04.335608 28653 sys_catalog.cc:455] T 00000000000000000000000000000000 P f89b8e74b0ae4f2daab1653a7fbb4e00 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f89b8e74b0ae4f2daab1653a7fbb4e00. Latest consensus state: current_term: 1 leader_uuid: "f89b8e74b0ae4f2daab1653a7fbb4e00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89b8e74b0ae4f2daab1653a7fbb4e00" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40437 } } }
I20260807 18:15:04.336076 28653 sys_catalog.cc:458] T 00000000000000000000000000000000 P f89b8e74b0ae4f2daab1653a7fbb4e00 [sys.catalog]: This master's current role is: LEADER
I20260807 18:15:04.337064 28652 sys_catalog.cc:455] T 00000000000000000000000000000000 P f89b8e74b0ae4f2daab1653a7fbb4e00 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f89b8e74b0ae4f2daab1653a7fbb4e00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89b8e74b0ae4f2daab1653a7fbb4e00" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40437 } } }
I20260807 18:15:04.337498 28652 sys_catalog.cc:458] T 00000000000000000000000000000000 P f89b8e74b0ae4f2daab1653a7fbb4e00 [sys.catalog]: This master's current role is: LEADER
I20260807 18:15:04.338764 28656 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:15:04.345748 28656 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:15:04.358078 28656 catalog_manager.cc:1347] Generated new cluster ID: 51864ea16ffa487a84653a42013b7eba
I20260807 18:15:04.358366 28656 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:15:04.377699 28656 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:15:04.379788 28656 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:15:04.396883 28656 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f89b8e74b0ae4f2daab1653a7fbb4e00: Generated new TSK 0
I20260807 18:15:04.397926 28656 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.17.110.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.110.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.110.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.1.keytab.
Entry for principal oryx/127.17.110.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.1.keytab.
I20260807 18:15:04.482445 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.1:0
--local_ip_for_outbound_sockets=127.17.110.1
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:40437
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.1.keytab
--principal=oryx/127.17.110.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:37387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:15:04.852380 28677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:04.852734 28677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:04.852911 28677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:04.861440 28677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:04.861584 28677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:04.861738 28677 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:15:04.861860 28677 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.1
I20260807 18:15:04.872536 28677 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:37387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.1.keytab
--principal=oryx/127.17.110.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.110.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.110.62:40437
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.1
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:15:04.873989 28677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:04.875483 28677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:04.890399 28682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:04.890600 28685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:04.890617 28683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:04.892033 28677 server_base.cc:1034] running on GCE node
I20260807 18:15:04.892741 28677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:04.893762 28677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:04.894995 28677 hybrid_clock.cc:648] HybridClock initialized: now 1786126504894918 us; error 51 us; skew 500 ppm
I20260807 18:15:04.895448 28677 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:15:04 dist-test-slave-39l8 krb5kdc[28513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126504, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:04.899166 28677 init.cc:377] Logged in from keytab as oryx/127.17.110.1@KRBTEST.COM (short username oryx)
I20260807 18:15:04.900889 28677 webserver.cc:460] Webserver started at http://127.17.110.1:45063/ using document root <none> and password file <none>
I20260807 18:15:04.901729 28677 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:04.901871 28677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:04.902233 28677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:15:04.905565 28677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-0/data/instance:
uuid: "f7cfb1ad38554e1b927a9dad4015e35d"
format_stamp: "Formatted at 2026-08-07 18:15:04 on dist-test-slave-39l8"
server_key: "d36d1dffcb865e72d86b50cdef432d2f"
server_key_iv: "ca982efd7685924c2329b5fad9d880f2"
server_key_version: "encryptionkey@0"
I20260807 18:15:04.906469 28677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance:
uuid: "f7cfb1ad38554e1b927a9dad4015e35d"
format_stamp: "Formatted at 2026-08-07 18:15:04 on dist-test-slave-39l8"
server_key: "d36d1dffcb865e72d86b50cdef432d2f"
server_key_iv: "ca982efd7685924c2329b5fad9d880f2"
server_key_version: "encryptionkey@0"
I20260807 18:15:04.912657 28677 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260807 18:15:04.917300 28697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:04.919155 28677 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 18:15:04.919399 28677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-0/wal
uuid: "f7cfb1ad38554e1b927a9dad4015e35d"
format_stamp: "Formatted at 2026-08-07 18:15:04 on dist-test-slave-39l8"
server_key: "d36d1dffcb865e72d86b50cdef432d2f"
server_key_iv: "ca982efd7685924c2329b5fad9d880f2"
server_key_version: "encryptionkey@0"
I20260807 18:15:04.919694 28677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:04.936421 28677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:04.942054 28677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:04.942394 28677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:04.945643 28677 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:15:04.945803 28677 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:04.945976 28677 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:15:04.946067 28677 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:15:04 dist-test-slave-39l8 krb5kdc[28513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126504, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM
I20260807 18:15:04.975416 28708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:04.946687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:57593 (local address 127.17.110.62:40437)
0807 18:15:04.947297 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:04.947302 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:04.949135 (+ 1833us) server_negotiation.cc:407] Connection header received
0807 18:15:04.951752 (+ 2617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:04.951758 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:04.951878 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:04.952039 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:04.955103 (+ 3064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:04.956327 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:04.959415 (+ 3088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:04.959910 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:04.960115 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:04.966423 (+ 6308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:04.966435 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:04.966444 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:04.966514 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:04.970270 (+ 3756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:04.971965 (+ 1695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:04.971975 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:04.971978 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:04.972181 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:04.973077 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:04.973084 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:04.973087 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:04.973431 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:04.973662 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:04.974622 (+ 960us) server_negotiation.cc:299] Negotiation successful
0807 18:15:04.974977 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":312,"threads_started":1}
I20260807 18:15:04.977571 28710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:04.947714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40437 (local address 127.17.110.1:57593)
0807 18:15:04.948859 (+ 1145us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:04.948910 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:04.951363 (+ 2453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:04.952379 (+ 1016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:04.952396 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:04.953482 (+ 1086us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:04.954834 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:04.954878 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:04.956540 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:04.956549 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:04.959176 (+ 2627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:04.959198 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:04.960242 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:04.960254 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:04.960655 (+ 401us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:04.961793 (+ 1138us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:04.961843 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:04.965943 (+ 4100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:04.970526 (+ 4583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:04.970545 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:04.970587 (+ 42us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:04.971707 (+ 1120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:04.972344 (+ 637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:04.972379 (+ 35us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:04.972387 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:04.972897 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:04.973620 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:04.973631 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:04.974126 (+ 495us) client_negotiation.cc:769] Sending connection context
0807 18:15:04.974456 (+ 330us) client_negotiation.cc:240] Negotiation successful
0807 18:15:04.975209 (+ 753us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":780,"thread_start_us":484,"threads_started":1}
I20260807 18:15:05.003129 28708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:04.985087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:41107 (local address 127.17.110.62:40437)
0807 18:15:04.985365 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:04.985377 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:04.985958 (+ 581us) server_negotiation.cc:407] Connection header received
0807 18:15:04.986142 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:04.986151 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:04.986283 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:04.986445 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:04.988277 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:04.989759 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:04.991697 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:04.992535 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:04.992730 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:04.994790 (+ 2060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:04.994798 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:04.994808 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:04.994863 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:04.999628 (+ 4765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:05.000574 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:05.000581 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:05.000585 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:05.000751 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:05.001236 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:05.001242 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:05.001248 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:05.001600 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:05.001807 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:05.002330 (+ 523us) server_negotiation.cc:299] Negotiation successful
0807 18:15:05.002654 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:15:05.003453 28806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:04.985085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40437 (local address 127.17.110.1:41107)
0807 18:15:04.985827 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:04.985854 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:04.985995 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:04.986502 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:04.986506 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:04.987181 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:04.987918 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:04.987941 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:04.989971 (+ 2030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:04.989979 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:04.991511 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:04.991527 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:04.992805 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:04.992811 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:04.992935 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:04.993686 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:04.993705 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:04.994484 (+ 779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:04.999844 (+ 5360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:04.999856 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:04.999860 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:05.000370 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:05.000899 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:05.000905 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:05.000910 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:05.001091 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:05.001751 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:05.001756 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:05.002016 (+ 260us) client_negotiation.cc:769] Sending connection context
0807 18:15:05.002201 (+ 185us) client_negotiation.cc:240] Negotiation successful
0807 18:15:05.002668 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":393,"threads_started":1}
I20260807 18:15:05.011364 28677 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.1:37549
I20260807 18:15:05.011466 28855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.1:37549 every 8 connection(s)
I20260807 18:15:05.013010 28677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
I20260807 18:15:05.015805 28856 heartbeater.cc:347] Connected to a master server at 127.17.110.62:40437
I20260807 18:15:05.016144 28856 heartbeater.cc:464] Registering TS with master...
I20260807 18:15:05.016896 28856 heartbeater.cc:511] Master 127.17.110.62:40437 requested a full tablet report, sending...
I20260807 18:15:05.019241 28570 ts_manager.cc:194] Registered new tserver with Master: f7cfb1ad38554e1b927a9dad4015e35d (127.17.110.1:37549)
I20260807 18:15:05.020834 28570 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.110.1@KRBTEST.COM'} at 127.17.110.1:57593
I20260807 18:15:05.022696 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 28677
I20260807 18:15:05.023010 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance
I20260807 18:15:05.023685 17848 external_mini_cluster.cc:1428] Setting key f94737d5e1ac7458f2417ae7c5690705
WARNING: no policy specified for oryx/127.17.110.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.110.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.110.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.2.keytab.
Entry for principal oryx/127.17.110.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.2.keytab.
I20260807 18:15:05.086894 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.2:0
--local_ip_for_outbound_sockets=127.17.110.2
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:40437
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.2.keytab
--principal=oryx/127.17.110.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:37387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:15:05.462507 28862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:05.462977 28862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:05.463169 28862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:05.470979 28862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:05.471158 28862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:05.471328 28862 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:15:05.471480 28862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.2
I20260807 18:15:05.482187 28862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:37387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.2.keytab
--principal=oryx/127.17.110.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.110.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.110.62:40437
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.2
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:15:05.483909 28862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:05.485672 28862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:05.500166 28868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:05.500205 28867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:05.500128 28870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:05.501338 28862 server_base.cc:1034] running on GCE node
I20260807 18:15:05.502198 28862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:05.503412 28862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:05.504680 28862 hybrid_clock.cc:648] HybridClock initialized: now 1786126505504564 us; error 86 us; skew 500 ppm
I20260807 18:15:05.505133 28862 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:15:05 dist-test-slave-39l8 krb5kdc[28513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126505, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:05.509250 28862 init.cc:377] Logged in from keytab as oryx/127.17.110.2@KRBTEST.COM (short username oryx)
I20260807 18:15:05.511117 28862 webserver.cc:460] Webserver started at http://127.17.110.2:35021/ using document root <none> and password file <none>
I20260807 18:15:05.511979 28862 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:05.512159 28862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:05.512571 28862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:15:05.516314 28862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-1/data/instance:
uuid: "bbc6335f70a54d61912f4ed30e2c6f79"
format_stamp: "Formatted at 2026-08-07 18:15:05 on dist-test-slave-39l8"
server_key: "6414bdd7f66c1682d247e3f1079c4b51"
server_key_iv: "9c3804b2738b53aabb0f61d304ec3207"
server_key_version: "encryptionkey@0"
I20260807 18:15:05.517330 28862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance:
uuid: "bbc6335f70a54d61912f4ed30e2c6f79"
format_stamp: "Formatted at 2026-08-07 18:15:05 on dist-test-slave-39l8"
server_key: "6414bdd7f66c1682d247e3f1079c4b51"
server_key_iv: "9c3804b2738b53aabb0f61d304ec3207"
server_key_version: "encryptionkey@0"
I20260807 18:15:05.524257 28862 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260807 18:15:05.528774 28877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:05.530546 28862 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260807 18:15:05.530853 28862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-1/wal
uuid: "bbc6335f70a54d61912f4ed30e2c6f79"
format_stamp: "Formatted at 2026-08-07 18:15:05 on dist-test-slave-39l8"
server_key: "6414bdd7f66c1682d247e3f1079c4b51"
server_key_iv: "9c3804b2738b53aabb0f61d304ec3207"
server_key_version: "encryptionkey@0"
I20260807 18:15:05.531152 28862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:05.553790 28862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:05.558406 28862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:05.558795 28862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:05.561882 28862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:15:05.561997 28862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:05.562165 28862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:15:05.562253 28862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:15:05 dist-test-slave-39l8 krb5kdc[28513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126505, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM
I20260807 18:15:05.593197 28890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:05.562560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:54791 (local address 127.17.110.62:40437)
0807 18:15:05.563371 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:05.563376 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:05.564509 (+ 1133us) server_negotiation.cc:407] Connection header received
0807 18:15:05.566981 (+ 2472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:05.566990 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:05.567137 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:05.567273 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:05.570327 (+ 3054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:05.571615 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:05.575343 (+ 3728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:05.575928 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:05.576120 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:05.582420 (+ 6300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:05.582431 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:05.582437 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:05.582519 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:05.587409 (+ 4890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:05.589118 (+ 1709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:05.589127 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:05.589132 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:05.589354 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:05.590207 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:05.590216 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:05.590222 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:05.590713 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:05.590954 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:05.592307 (+ 1353us) server_negotiation.cc:299] Negotiation successful
0807 18:15:05.592654 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":570,"threads_started":1}
I20260807 18:15:05.595368 28891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:05.563346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40437 (local address 127.17.110.2:54791)
0807 18:15:05.564297 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:05.564357 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:05.566572 (+ 2215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:05.567666 (+ 1094us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:05.567683 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:05.568835 (+ 1152us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:05.570084 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:05.570111 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:05.571880 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:05.571893 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:05.575058 (+ 3165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:05.575088 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:05.576283 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:05.576296 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:05.576732 (+ 436us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:05.577795 (+ 1063us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:05.577849 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:05.582004 (+ 4155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:05.587774 (+ 5770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:05.587794 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:05.587824 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:05.588869 (+ 1045us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:05.589538 (+ 669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:05.589547 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:05.589554 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:05.590018 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:05.590928 (+ 910us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:05.590943 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:05.591657 (+ 714us) client_negotiation.cc:769] Sending connection context
0807 18:15:05.592138 (+ 481us) client_negotiation.cc:240] Negotiation successful
0807 18:15:05.593021 (+ 883us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":417,"threads_started":1}
I20260807 18:15:05.608727 28862 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.2:37795
I20260807 18:15:05.608834 29049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.2:37795 every 8 connection(s)
I20260807 18:15:05.610174 28862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
I20260807 18:15:05.612707 29051 heartbeater.cc:347] Connected to a master server at 127.17.110.62:40437
I20260807 18:15:05.612982 29051 heartbeater.cc:464] Registering TS with master...
I20260807 18:15:05.613767 29051 heartbeater.cc:511] Master 127.17.110.62:40437 requested a full tablet report, sending...
I20260807 18:15:05.615633 28570 ts_manager.cc:194] Registered new tserver with Master: bbc6335f70a54d61912f4ed30e2c6f79 (127.17.110.2:37795)
I20260807 18:15:05.616786 28570 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.110.2@KRBTEST.COM'} at 127.17.110.2:54791
I20260807 18:15:05.618510 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 28862
I20260807 18:15:05.618825 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance
I20260807 18:15:05.619508 17848 external_mini_cluster.cc:1428] Setting key 4e3e97fddc463ca8f86dc9db2db6617b
I20260807 18:15:05.621855 28890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:05.605059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:59719 (local address 127.17.110.62:40437)
0807 18:15:05.605270 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:05.605275 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:05.606411 (+ 1136us) server_negotiation.cc:407] Connection header received
0807 18:15:05.606751 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:05.606757 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:05.606859 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:05.606981 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:05.608368 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:05.609384 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:05.610923 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:05.611558 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:05.611697 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:05.614189 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:05.614196 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:05.614200 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:05.614247 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:05.618083 (+ 3836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:05.619136 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:05.619142 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:05.619146 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:05.619364 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:05.619851 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:05.619856 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:05.619860 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:05.620515 (+ 655us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:05.620720 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:05.621236 (+ 516us) server_negotiation.cc:299] Negotiation successful
0807 18:15:05.621501 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 18:15:05.621930 29036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:05.605254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40437 (local address 127.17.110.2:59719)
0807 18:15:05.606257 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:05.606281 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:05.606560 (+ 279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:05.607018 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:05.607023 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:05.607765 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:05.608191 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:05.608203 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:05.609536 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:05.609542 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:05.610648 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:05.610683 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:05.611994 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:05.611999 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:05.612149 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:05.612954 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:05.612977 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:05.613957 (+ 980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:05.618279 (+ 4322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:05.618286 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:05.618289 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:05.618948 (+ 659us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:05.619513 (+ 565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:05.619518 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:05.619522 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:05.619697 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:05.620677 (+ 980us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:05.620681 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:05.620865 (+ 184us) client_negotiation.cc:769] Sending connection context
0807 18:15:05.621061 (+ 196us) client_negotiation.cc:240] Negotiation successful
0807 18:15:05.621451 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":780,"thread_start_us":584,"threads_started":1}
WARNING: no policy specified for oryx/127.17.110.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.110.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.110.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.3.keytab.
Entry for principal oryx/127.17.110.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.3.keytab.
I20260807 18:15:05.680612 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.3:0
--local_ip_for_outbound_sockets=127.17.110.3
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:40437
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.3.keytab
--principal=oryx/127.17.110.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:37387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:15:06.024658 28856 heartbeater.cc:503] Master 127.17.110.62:40437 was elected leader, sending a full tablet report...
W20260807 18:15:06.059684 29056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:06.060065 29056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:06.060262 29056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:06.068045 29056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:06.068251 29056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:06.068408 29056 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:15:06.068548 29056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.3
I20260807 18:15:06.079126 29056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:37387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.3.keytab
--principal=oryx/127.17.110.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.110.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.110.62:40437
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.3
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:15:06.080708 29056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:06.082319 29056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:06.098910 29063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:06.098984 29068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:06.098929 29065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:06.100256 29056 server_base.cc:1034] running on GCE node
I20260807 18:15:06.101114 29056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:06.102312 29056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:06.103636 29056 hybrid_clock.cc:648] HybridClock initialized: now 1786126506103534 us; error 70 us; skew 500 ppm
I20260807 18:15:06.104066 29056 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:15:06 dist-test-slave-39l8 krb5kdc[28513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126506, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:06.108144 29056 init.cc:377] Logged in from keytab as oryx/127.17.110.3@KRBTEST.COM (short username oryx)
I20260807 18:15:06.110141 29056 webserver.cc:460] Webserver started at http://127.17.110.3:36423/ using document root <none> and password file <none>
I20260807 18:15:06.111120 29056 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:06.111234 29056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:06.111545 29056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:15:06.114902 29056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-2/data/instance:
uuid: "62030defec5b46a58dd672eeb50f5049"
format_stamp: "Formatted at 2026-08-07 18:15:06 on dist-test-slave-39l8"
server_key: "21b1eb571dfb839cc926d95319acf99e"
server_key_iv: "b7b0a7e0684a8b277aa71eb0a85baf24"
server_key_version: "encryptionkey@0"
I20260807 18:15:06.115828 29056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance:
uuid: "62030defec5b46a58dd672eeb50f5049"
format_stamp: "Formatted at 2026-08-07 18:15:06 on dist-test-slave-39l8"
server_key: "21b1eb571dfb839cc926d95319acf99e"
server_key_iv: "b7b0a7e0684a8b277aa71eb0a85baf24"
server_key_version: "encryptionkey@0"
I20260807 18:15:06.122082 29056 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 18:15:06.126758 29079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:06.128607 29056 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:15:06.128841 29056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-2/wal
uuid: "62030defec5b46a58dd672eeb50f5049"
format_stamp: "Formatted at 2026-08-07 18:15:06 on dist-test-slave-39l8"
server_key: "21b1eb571dfb839cc926d95319acf99e"
server_key_iv: "b7b0a7e0684a8b277aa71eb0a85baf24"
server_key_version: "encryptionkey@0"
I20260807 18:15:06.129186 29056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:06.182001 29056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:06.187291 29056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:06.187614 29056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:06.190647 29056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:15:06.190828 29056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260807 18:15:06.191016 29056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:15:06.191089 29056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:15:06 dist-test-slave-39l8 krb5kdc[28513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126506, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM
I20260807 18:15:06.222558 29091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:06.191741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:56763 (local address 127.17.110.62:40437)
0807 18:15:06.192461 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:06.192466 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:06.193792 (+ 1326us) server_negotiation.cc:407] Connection header received
0807 18:15:06.196276 (+ 2484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:06.196282 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:06.196396 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:06.196541 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:06.200011 (+ 3470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:06.201335 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:06.205079 (+ 3744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:06.205881 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:06.206067 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:06.213284 (+ 7217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:06.213293 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:06.213298 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:06.213353 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:06.217147 (+ 3794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:06.218294 (+ 1147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:06.218304 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:06.218311 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:06.218520 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:06.219476 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:06.219484 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:06.219489 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:06.219986 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:06.220186 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:06.221671 (+ 1485us) server_negotiation.cc:299] Negotiation successful
0807 18:15:06.222010 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":441,"threads_started":1}
I20260807 18:15:06.225791 29092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:06.192530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40437 (local address 127.17.110.3:56763)
0807 18:15:06.193569 (+ 1039us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:06.193628 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:06.195917 (+ 2289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:06.196970 (+ 1053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:06.196989 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:06.198390 (+ 1401us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:06.199697 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:06.199744 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:06.201626 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:06.201637 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:06.204803 (+ 3166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:06.204833 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:06.206340 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:06.206354 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:06.206882 (+ 528us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:06.208023 (+ 1141us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:06.208089 (+ 66us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:06.212807 (+ 4718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:06.217373 (+ 4566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:06.217391 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:06.217421 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:06.218098 (+ 677us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:06.218763 (+ 665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:06.218773 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:06.218778 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:06.219266 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:06.220186 (+ 920us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:06.220201 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:06.221023 (+ 822us) client_negotiation.cc:769] Sending connection context
0807 18:15:06.221545 (+ 522us) client_negotiation.cc:240] Negotiation successful
0807 18:15:06.223245 (+ 1700us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"thread_start_us":416,"threads_started":1}
I20260807 18:15:06.240302 29056 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.3:36959
I20260807 18:15:06.240325 29244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.3:36959 every 8 connection(s)
I20260807 18:15:06.241825 29056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
I20260807 18:15:06.243858 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 29056
I20260807 18:15:06.244065 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance
I20260807 18:15:06.244715 17848 external_mini_cluster.cc:1428] Setting key 0b9bc17d37d1a9b6e30cf3793386d3b4
I20260807 18:15:06.246624 29245 heartbeater.cc:347] Connected to a master server at 127.17.110.62:40437
I20260807 18:15:06.246966 29245 heartbeater.cc:464] Registering TS with master...
I20260807 18:15:06.247912 29245 heartbeater.cc:511] Master 127.17.110.62:40437 requested a full tablet report, sending...
I20260807 18:15:06.249562 28569 ts_manager.cc:194] Registered new tserver with Master: 62030defec5b46a58dd672eeb50f5049 (127.17.110.3:36959)
I20260807 18:15:06.250921 28569 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.110.3@KRBTEST.COM'} at 127.17.110.3:56763
I20260807 18:15:06.252069 29091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:06.235135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:58849 (local address 127.17.110.62:40437)
0807 18:15:06.235356 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:06.235364 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:06.236319 (+ 955us) server_negotiation.cc:407] Connection header received
0807 18:15:06.236534 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:06.236539 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:06.236676 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:06.236834 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:06.237888 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:06.238907 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:06.240903 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:06.241409 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:06.241582 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:06.244520 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:06.244525 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:06.244528 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:06.244565 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:06.248609 (+ 4044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:06.249708 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:06.249713 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:06.249716 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:06.249906 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:06.250407 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:06.250412 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:06.250415 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:06.250776 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:06.250960 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:06.251450 (+ 490us) server_negotiation.cc:299] Negotiation successful
0807 18:15:06.251722 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:15:06.252148 29236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:06.235170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40437 (local address 127.17.110.3:58849)
0807 18:15:06.236137 (+ 967us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:06.236169 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:06.236381 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:06.236875 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:06.236880 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:06.237408 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:06.237719 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:06.237729 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:06.239129 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:06.239135 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:06.240714 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:06.240729 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:06.241627 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:06.241635 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:06.241828 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:06.242680 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:06.242803 (+ 123us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:06.244256 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:06.249016 (+ 4760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:06.249025 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:06.249037 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:06.249558 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:06.250130 (+ 572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:06.250134 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:06.250137 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:06.250276 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:06.250937 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:06.250941 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:06.251180 (+ 239us) client_negotiation.cc:769] Sending connection context
0807 18:15:06.251322 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 18:15:06.251713 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"thread_start_us":535,"threads_started":1}
I20260807 18:15:06.261981 17848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:15:06 dist-test-slave-39l8 krb5kdc[28513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126506, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 18:15:06 dist-test-slave-39l8 krb5kdc[28513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:06 dist-test-slave-39l8 krb5kdc[28513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:06.620637 29051 heartbeater.cc:503] Master 127.17.110.62:40437 was elected leader, sending a full tablet report...
I20260807 18:15:07.254987 29245 heartbeater.cc:503] Master 127.17.110.62:40437 was elected leader, sending a full tablet report...
W20260807 18:15:07.506896 29256 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.175s user 0.002s sys 0.020s
W20260807 18:15:07.508157 29091 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:06.324509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50776 (local address 127.17.110.62:40437)
0807 18:15:06.324796 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:06.324803 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:06.325019 (+ 216us) server_negotiation.cc:407] Connection header received
0807 18:15:06.325270 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:06.325277 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:06.325363 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:06.325479 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:06.327252 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:06.328198 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:06.330167 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:06.330652 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:06.330878 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:07.507865 (+1176987us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50776: BlockingRecv error: recv got EOF from 127.0.0.1:50776 (error 108)
Metrics: {"server-negotiator.queue_time_us":146}
Aug 07 18:15:07 dist-test-slave-39l8 krb5kdc[28513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126506, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM
I20260807 18:15:07.529345 29091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:07.513077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50780 (local address 127.17.110.62:40437)
0807 18:15:07.513380 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:07.513387 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:07.513691 (+ 304us) server_negotiation.cc:407] Connection header received
0807 18:15:07.513845 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:07.513849 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:07.513919 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:07.514029 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:07.515592 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:07.516381 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:07.517976 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:07.518467 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:07.518586 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:07.523213 (+ 4627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:07.523218 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:07.523222 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:07.523310 (+ 88us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:07.526301 (+ 2991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:07.527168 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:07.527173 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:07.527175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:07.527391 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:07.527885 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:07.527889 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:07.527891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:07.528228 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:07.528598 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:07.528781 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 18:15:07.529053 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260807 18:15:07.534483 28569 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50780:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:15:07.542294 28569 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:15:07.568385 29274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:07.557408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40028 (local address 127.17.110.2:37795)
0807 18:15:07.558428 (+ 1020us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:07.558438 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:07.559055 (+ 617us) server_negotiation.cc:407] Connection header received
0807 18:15:07.559348 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:07.559355 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:07.559532 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:07.559704 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:15:07.561038 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:07.562807 (+ 1769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:07.565523 (+ 2716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:07.567107 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:07.567285 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:07.567531 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:07.567869 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 18:15:07.568052 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":870,"thread_start_us":736,"threads_started":1}
I20260807 18:15:07.568490 29273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:07.556599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:37795 (local address 127.0.0.1:40028)
0807 18:15:07.558941 (+ 2342us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:07.558960 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:07.559169 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:07.559969 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:07.559978 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:07.560033 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:15:07.560807 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:07.560821 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:07.562967 (+ 2146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:07.562973 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:07.565240 (+ 2267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:07.565308 (+ 68us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:07.567349 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:07.567356 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:07.567591 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:07.567653 (+ 62us) client_negotiation.cc:769] Sending connection context
0807 18:15:07.567813 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 18:15:07.568038 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2211,"mutex_wait_us":183,"spinlock_wait_cycles":17408,"thread_start_us":2030,"threads_started":1}
I20260807 18:15:07.569548 29272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:07.556502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56400 (local address 127.17.110.1:37549)
0807 18:15:07.558207 (+ 1705us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:07.558219 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:07.559116 (+ 897us) server_negotiation.cc:407] Connection header received
0807 18:15:07.559229 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:07.559238 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:07.559514 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:07.559710 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:15:07.560944 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:07.562806 (+ 1862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:07.566179 (+ 3373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:07.567940 (+ 1761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:07.568123 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:07.568390 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:07.569158 (+ 768us) server_negotiation.cc:299] Negotiation successful
0807 18:15:07.569319 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1445,"thread_start_us":1307,"threads_started":1}
I20260807 18:15:07.569689 29271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:07.556272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:37549 (local address 127.0.0.1:56400)
0807 18:15:07.558525 (+ 2253us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:07.558585 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:07.558865 (+ 280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:07.559969 (+ 1104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:07.559978 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:07.560034 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:15:07.560735 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:07.560753 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:07.563032 (+ 2279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:07.563039 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:07.565913 (+ 2874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:07.565931 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:07.568628 (+ 2697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:07.568638 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:07.568898 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:07.568944 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 18:15:07.569074 (+ 130us) client_negotiation.cc:240] Negotiation successful
0807 18:15:07.569250 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1202,"mutex_wait_us":141,"spinlock_wait_cycles":4224,"thread_start_us":1013,"threads_started":1}
I20260807 18:15:07.572139 29276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:07.559617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48250 (local address 127.17.110.3:36959)
0807 18:15:07.560652 (+ 1035us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:07.560660 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:07.560690 (+ 30us) server_negotiation.cc:407] Connection header received
0807 18:15:07.560792 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:07.560799 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:07.561041 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:07.561248 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:15:07.561895 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:07.563418 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:07.566385 (+ 2967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:07.568084 (+ 1699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:07.568242 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:07.568452 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:07.571632 (+ 3180us) server_negotiation.cc:299] Negotiation successful
0807 18:15:07.571841 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":847,"thread_start_us":708,"threads_started":1}
I20260807 18:15:07.572731 29275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:07.559020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:36959 (local address 127.0.0.1:48250)
0807 18:15:07.559717 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:07.559738 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:07.559873 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:07.561235 (+ 1362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:07.561240 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:07.561266 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:15:07.561678 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:07.561692 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:07.563593 (+ 1901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:07.563599 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:07.566144 (+ 2545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:07.566159 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:07.568647 (+ 2488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:07.568655 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:07.571391 (+ 2736us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:07.571445 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 18:15:07.571552 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 18:15:07.571713 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"mutex_wait_us":1,"thread_start_us":811,"threads_started":1}
I20260807 18:15:07.573637 28758 tablet_service.cc:1467] Processing CreateTablet for tablet ecef5162fab040fb89b9a539d58c67c4 (DEFAULT_TABLE table=test-table [id=bcda2ad50dbb4ac290284676b1d1f003]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:15:07.573637 28945 tablet_service.cc:1467] Processing CreateTablet for tablet ecef5162fab040fb89b9a539d58c67c4 (DEFAULT_TABLE table=test-table [id=bcda2ad50dbb4ac290284676b1d1f003]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:15:07.575264 28758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecef5162fab040fb89b9a539d58c67c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:07.575263 28945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecef5162fab040fb89b9a539d58c67c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:07.576328 29146 tablet_service.cc:1467] Processing CreateTablet for tablet ecef5162fab040fb89b9a539d58c67c4 (DEFAULT_TABLE table=test-table [id=bcda2ad50dbb4ac290284676b1d1f003]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:15:07.578475 29146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecef5162fab040fb89b9a539d58c67c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:07.592330 29278 tablet_bootstrap.cc:492] T ecef5162fab040fb89b9a539d58c67c4 P f7cfb1ad38554e1b927a9dad4015e35d: Bootstrap starting.
I20260807 18:15:07.594094 29277 tablet_bootstrap.cc:492] T ecef5162fab040fb89b9a539d58c67c4 P 62030defec5b46a58dd672eeb50f5049: Bootstrap starting.
I20260807 18:15:07.596158 29278 tablet_bootstrap.cc:654] T ecef5162fab040fb89b9a539d58c67c4 P f7cfb1ad38554e1b927a9dad4015e35d: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:07.597340 29278 log.cc:824] T ecef5162fab040fb89b9a539d58c67c4 P f7cfb1ad38554e1b927a9dad4015e35d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:15:07.597534 29279 tablet_bootstrap.cc:492] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79: Bootstrap starting.
I20260807 18:15:07.598611 29277 tablet_bootstrap.cc:654] T ecef5162fab040fb89b9a539d58c67c4 P 62030defec5b46a58dd672eeb50f5049: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:07.599979 29277 log.cc:824] T ecef5162fab040fb89b9a539d58c67c4 P 62030defec5b46a58dd672eeb50f5049: Log is configured to *not* fsync() on all Append() calls
I20260807 18:15:07.600769 29278 tablet_bootstrap.cc:492] T ecef5162fab040fb89b9a539d58c67c4 P f7cfb1ad38554e1b927a9dad4015e35d: No bootstrap required, opened a new log
I20260807 18:15:07.601085 29278 ts_tablet_manager.cc:1397] T ecef5162fab040fb89b9a539d58c67c4 P f7cfb1ad38554e1b927a9dad4015e35d: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260807 18:15:07.603940 29277 tablet_bootstrap.cc:492] T ecef5162fab040fb89b9a539d58c67c4 P 62030defec5b46a58dd672eeb50f5049: No bootstrap required, opened a new log
I20260807 18:15:07.604265 29277 ts_tablet_manager.cc:1397] T ecef5162fab040fb89b9a539d58c67c4 P 62030defec5b46a58dd672eeb50f5049: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260807 18:15:07.605540 29279 tablet_bootstrap.cc:654] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:07.606685 29279 log.cc:824] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79: Log is configured to *not* fsync() on all Append() calls
I20260807 18:15:07.610287 29279 tablet_bootstrap.cc:492] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79: No bootstrap required, opened a new log
I20260807 18:15:07.610662 29279 ts_tablet_manager.cc:1397] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.004s
I20260807 18:15:07.612702 29278 raft_consensus.cc:348] T ecef5162fab040fb89b9a539d58c67c4 P f7cfb1ad38554e1b927a9dad4015e35d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:07.613356 29278 raft_consensus.cc:374] T ecef5162fab040fb89b9a539d58c67c4 P f7cfb1ad38554e1b927a9dad4015e35d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:07.613512 29278 raft_consensus.cc:729] T ecef5162fab040fb89b9a539d58c67c4 P f7cfb1ad38554e1b927a9dad4015e35d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7cfb1ad38554e1b927a9dad4015e35d, State: Initialized, Role: FOLLOWER
I20260807 18:15:07.614459 29278 consensus_queue.cc:260] T ecef5162fab040fb89b9a539d58c67c4 P f7cfb1ad38554e1b927a9dad4015e35d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:07.616324 29277 raft_consensus.cc:348] T ecef5162fab040fb89b9a539d58c67c4 P 62030defec5b46a58dd672eeb50f5049 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:07.617024 29277 raft_consensus.cc:374] T ecef5162fab040fb89b9a539d58c67c4 P 62030defec5b46a58dd672eeb50f5049 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:07.617201 29277 raft_consensus.cc:729] T ecef5162fab040fb89b9a539d58c67c4 P 62030defec5b46a58dd672eeb50f5049 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62030defec5b46a58dd672eeb50f5049, State: Initialized, Role: FOLLOWER
I20260807 18:15:07.618321 29277 consensus_queue.cc:260] T ecef5162fab040fb89b9a539d58c67c4 P 62030defec5b46a58dd672eeb50f5049 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:07.622329 29279 raft_consensus.cc:348] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:07.622932 29279 raft_consensus.cc:374] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:07.623083 29279 raft_consensus.cc:729] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbc6335f70a54d61912f4ed30e2c6f79, State: Initialized, Role: FOLLOWER
I20260807 18:15:07.623555 29278 ts_tablet_manager.cc:1428] T ecef5162fab040fb89b9a539d58c67c4 P f7cfb1ad38554e1b927a9dad4015e35d: Time spent starting tablet: real 0.022s user 0.017s sys 0.005s
I20260807 18:15:07.624820 29279 consensus_queue.cc:260] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:07.625522 29277 ts_tablet_manager.cc:1428] T ecef5162fab040fb89b9a539d58c67c4 P 62030defec5b46a58dd672eeb50f5049: Time spent starting tablet: real 0.021s user 0.021s sys 0.000s
I20260807 18:15:07.628593 29279 ts_tablet_manager.cc:1428] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79: Time spent starting tablet: real 0.018s user 0.012s sys 0.003s
W20260807 18:15:07.745177 29248 tablet.cc:2369] T ecef5162fab040fb89b9a539d58c67c4 P 62030defec5b46a58dd672eeb50f5049: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:15:07.767117 28859 tablet.cc:2369] T ecef5162fab040fb89b9a539d58c67c4 P f7cfb1ad38554e1b927a9dad4015e35d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:15:07.864169 29052 tablet.cc:2369] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:15:07.964291 29288 raft_consensus.cc:482] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:15:07.964661 29288 raft_consensus.cc:504] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:07.966646 29288 leader_election.cc:288] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7cfb1ad38554e1b927a9dad4015e35d (127.17.110.1:37549), 62030defec5b46a58dd672eeb50f5049 (127.17.110.3:36959)
I20260807 18:15:07.974680 29272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:07.967197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:46233 (local address 127.17.110.1:37549)
0807 18:15:07.967535 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:07.967545 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:07.968153 (+ 608us) server_negotiation.cc:407] Connection header received
0807 18:15:07.968451 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:07.968455 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:07.968518 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:07.968590 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:15:07.969275 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:07.970063 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:07.972439 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:07.973359 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:07.973511 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:07.973576 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:07.974242 (+ 666us) server_negotiation.cc:299] Negotiation successful
0807 18:15:07.974417 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177}
I20260807 18:15:07.974852 29276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:07.967286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:39651 (local address 127.17.110.3:36959)
0807 18:15:07.967494 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:07.967501 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:07.968181 (+ 680us) server_negotiation.cc:407] Connection header received
0807 18:15:07.968499 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:07.968503 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:07.968593 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:07.968696 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:15:07.969533 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:07.970465 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:07.972617 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:07.973624 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:07.973831 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:07.973914 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:07.974415 (+ 501us) server_negotiation.cc:299] Negotiation successful
0807 18:15:07.974580 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 18:15:07.974917 29291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:07.966948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:37549 (local address 127.17.110.2:46233)
0807 18:15:07.967997 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:07.968020 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:07.968244 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:07.968653 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:07.968660 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:07.968690 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:15:07.969096 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:07.969109 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:07.970246 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:07.970253 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:07.972206 (+ 1953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:07.972226 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:07.973739 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:07.973748 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:07.974028 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:07.974059 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 18:15:07.974151 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 18:15:07.974309 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":862,"mutex_wait_us":53,"thread_start_us":576,"threads_started":1}
I20260807 18:15:07.975219 29292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:07.967212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:36959 (local address 127.17.110.2:39651)
0807 18:15:07.968049 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:07.968065 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:07.968298 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:07.968741 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:07.968746 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:07.968815 (+ 69us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:15:07.969229 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:07.969242 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:07.970700 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:07.970705 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:07.972384 (+ 1679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:07.972403 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:07.973897 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:07.973903 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:07.974100 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:07.974163 (+ 63us) client_negotiation.cc:769] Sending connection context
0807 18:15:07.974359 (+ 196us) client_negotiation.cc:240] Negotiation successful
0807 18:15:07.974501 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"mutex_wait_us":45,"spinlock_wait_cycles":85504,"thread_start_us":535,"threads_started":1}
I20260807 18:15:07.976265 28788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecef5162fab040fb89b9a539d58c67c4" candidate_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" is_pre_election: true
I20260807 18:15:07.976435 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecef5162fab040fb89b9a539d58c67c4" candidate_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62030defec5b46a58dd672eeb50f5049" is_pre_election: true
I20260807 18:15:07.976863 28788 raft_consensus.cc:2454] T ecef5162fab040fb89b9a539d58c67c4 P f7cfb1ad38554e1b927a9dad4015e35d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbc6335f70a54d61912f4ed30e2c6f79 in term 0.
I20260807 18:15:07.977092 29175 raft_consensus.cc:2454] T ecef5162fab040fb89b9a539d58c67c4 P 62030defec5b46a58dd672eeb50f5049 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbc6335f70a54d61912f4ed30e2c6f79 in term 0.
I20260807 18:15:07.977757 28885 leader_election.cc:302] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 62030defec5b46a58dd672eeb50f5049, bbc6335f70a54d61912f4ed30e2c6f79, f7cfb1ad38554e1b927a9dad4015e35d; no voters:
I20260807 18:15:07.978279 29288 raft_consensus.cc:2781] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:15:07.978456 29288 raft_consensus.cc:482] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:15:07.978577 29288 raft_consensus.cc:3037] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:07.981073 29288 raft_consensus.cc:504] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:07.981916 29288 leader_election.cc:288] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79 [CANDIDATE]: Term 1 election: Requested vote from peers f7cfb1ad38554e1b927a9dad4015e35d (127.17.110.1:37549), 62030defec5b46a58dd672eeb50f5049 (127.17.110.3:36959)
I20260807 18:15:07.982527 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecef5162fab040fb89b9a539d58c67c4" candidate_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62030defec5b46a58dd672eeb50f5049"
I20260807 18:15:07.982519 28788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecef5162fab040fb89b9a539d58c67c4" candidate_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7cfb1ad38554e1b927a9dad4015e35d"
I20260807 18:15:07.982788 29175 raft_consensus.cc:3037] T ecef5162fab040fb89b9a539d58c67c4 P 62030defec5b46a58dd672eeb50f5049 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:07.982803 28788 raft_consensus.cc:3037] T ecef5162fab040fb89b9a539d58c67c4 P f7cfb1ad38554e1b927a9dad4015e35d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:07.985181 29175 raft_consensus.cc:2454] T ecef5162fab040fb89b9a539d58c67c4 P 62030defec5b46a58dd672eeb50f5049 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbc6335f70a54d61912f4ed30e2c6f79 in term 1.
I20260807 18:15:07.985402 28788 raft_consensus.cc:2454] T ecef5162fab040fb89b9a539d58c67c4 P f7cfb1ad38554e1b927a9dad4015e35d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbc6335f70a54d61912f4ed30e2c6f79 in term 1.
I20260807 18:15:07.985751 28884 leader_election.cc:302] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62030defec5b46a58dd672eeb50f5049, bbc6335f70a54d61912f4ed30e2c6f79; no voters:
I20260807 18:15:07.986136 29288 raft_consensus.cc:2781] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:15:07.986647 29288 raft_consensus.cc:686] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79 [term 1 LEADER]: Becoming Leader. State: Replica: bbc6335f70a54d61912f4ed30e2c6f79, State: Running, Role: LEADER
I20260807 18:15:07.987350 29288 consensus_queue.cc:237] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:07.992682 28568 catalog_manager.cc:5495] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79 reported cstate change: term changed from 0 to 1, leader changed from <none> to bbc6335f70a54d61912f4ed30e2c6f79 (127.17.110.2). New cstate: current_term: 1 leader_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } health_report { overall_health: UNKNOWN } } }
I20260807 18:15:08.037480 29274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:08.030862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40042 (local address 127.17.110.2:37795)
0807 18:15:08.031127 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:08.031134 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:08.031310 (+ 176us) server_negotiation.cc:407] Connection header received
0807 18:15:08.031593 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:08.031598 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:08.031699 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:08.031873 (+ 174us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:15:08.032366 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.033094 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.034630 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.035210 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.035359 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:08.035861 (+ 502us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:15:08.036564 (+ 703us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:15:08.036907 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:08.036985 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 18:15:08.037142 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:15:08.047011 29175 raft_consensus.cc:1261] T ecef5162fab040fb89b9a539d58c67c4 P 62030defec5b46a58dd672eeb50f5049 [term 1 FOLLOWER]: Refusing update from remote peer bbc6335f70a54d61912f4ed30e2c6f79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:15:08.047210 28788 raft_consensus.cc:1261] T ecef5162fab040fb89b9a539d58c67c4 P f7cfb1ad38554e1b927a9dad4015e35d [term 1 FOLLOWER]: Refusing update from remote peer bbc6335f70a54d61912f4ed30e2c6f79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:15:08.048187 29294 consensus_queue.cc:1035] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:15:08.048605 29288 consensus_queue.cc:1035] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:15:08.063525 29304 mvcc.cc:205] Tried to move back new op lower bound from 7315974176948412416 to 7315974176720334848. Current Snapshot: MvccSnapshot[applied={T|T < 7315974176948412416}]
I20260807 18:15:08.080590 28570 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:15:08.080884 28570 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:15:08.085139 28570 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f89b8e74b0ae4f2daab1653a7fbb4e00: Sending DeleteTablet for 3 replicas of tablet ecef5162fab040fb89b9a539d58c67c4
I20260807 18:15:08.086651 28758 tablet_service.cc:1514] Processing DeleteTablet for tablet ecef5162fab040fb89b9a539d58c67c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:15:08 UTC) from {username='oryx', principal='oryx/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:56400
I20260807 18:15:08.086653 28945 tablet_service.cc:1514] Processing DeleteTablet for tablet ecef5162fab040fb89b9a539d58c67c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:15:08 UTC) from {username='oryx', principal='oryx/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:40028
I20260807 18:15:08.087589 29146 tablet_service.cc:1514] Processing DeleteTablet for tablet ecef5162fab040fb89b9a539d58c67c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:15:08 UTC) from {username='oryx', principal='oryx/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:48250
I20260807 18:15:08.088683 28568 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50780:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:15:08.089463 28568 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:15:08.096771 29319 tablet_replica.cc:330] T ecef5162fab040fb89b9a539d58c67c4 P 62030defec5b46a58dd672eeb50f5049: stopping tablet replica
I20260807 18:15:08.099435 29319 raft_consensus.cc:2229] T ecef5162fab040fb89b9a539d58c67c4 P 62030defec5b46a58dd672eeb50f5049 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:15:08.100508 29319 raft_consensus.cc:2258] T ecef5162fab040fb89b9a539d58c67c4 P 62030defec5b46a58dd672eeb50f5049 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:15:08.102878 28757 tablet_service.cc:1467] Processing CreateTablet for tablet 979aa541a753448c8d6d9ab6b68d0d04 (DEFAULT_TABLE table=test-table [id=72845f2d237c4102a39d1d78e5ad3e55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:15:08.103281 29146 tablet_service.cc:1467] Processing CreateTablet for tablet 979aa541a753448c8d6d9ab6b68d0d04 (DEFAULT_TABLE table=test-table [id=72845f2d237c4102a39d1d78e5ad3e55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:15:08.103483 28757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 979aa541a753448c8d6d9ab6b68d0d04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:08.103832 29146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 979aa541a753448c8d6d9ab6b68d0d04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:08.103878 28944 tablet_service.cc:1467] Processing CreateTablet for tablet 979aa541a753448c8d6d9ab6b68d0d04 (DEFAULT_TABLE table=test-table [id=72845f2d237c4102a39d1d78e5ad3e55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:15:08.104526 28944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 979aa541a753448c8d6d9ab6b68d0d04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:08.107810 29319 ts_tablet_manager.cc:1905] T ecef5162fab040fb89b9a539d58c67c4 P 62030defec5b46a58dd672eeb50f5049: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:15:08.110626 29278 tablet_bootstrap.cc:492] T 979aa541a753448c8d6d9ab6b68d0d04 P f7cfb1ad38554e1b927a9dad4015e35d: Bootstrap starting.
I20260807 18:15:08.111409 29277 tablet_bootstrap.cc:492] T 979aa541a753448c8d6d9ab6b68d0d04 P 62030defec5b46a58dd672eeb50f5049: Bootstrap starting.
I20260807 18:15:08.112555 29278 tablet_bootstrap.cc:654] T 979aa541a753448c8d6d9ab6b68d0d04 P f7cfb1ad38554e1b927a9dad4015e35d: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:08.113247 29277 tablet_bootstrap.cc:654] T 979aa541a753448c8d6d9ab6b68d0d04 P 62030defec5b46a58dd672eeb50f5049: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:08.116918 29279 tablet_bootstrap.cc:492] T 979aa541a753448c8d6d9ab6b68d0d04 P bbc6335f70a54d61912f4ed30e2c6f79: Bootstrap starting.
I20260807 18:15:08.118937 29279 tablet_bootstrap.cc:654] T 979aa541a753448c8d6d9ab6b68d0d04 P bbc6335f70a54d61912f4ed30e2c6f79: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:08.123607 29278 tablet_bootstrap.cc:492] T 979aa541a753448c8d6d9ab6b68d0d04 P f7cfb1ad38554e1b927a9dad4015e35d: No bootstrap required, opened a new log
I20260807 18:15:08.123962 29278 ts_tablet_manager.cc:1397] T 979aa541a753448c8d6d9ab6b68d0d04 P f7cfb1ad38554e1b927a9dad4015e35d: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.006s
I20260807 18:15:08.125341 29278 raft_consensus.cc:348] T 979aa541a753448c8d6d9ab6b68d0d04 P f7cfb1ad38554e1b927a9dad4015e35d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:08.125850 29278 raft_consensus.cc:374] T 979aa541a753448c8d6d9ab6b68d0d04 P f7cfb1ad38554e1b927a9dad4015e35d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:08.126017 29278 raft_consensus.cc:729] T 979aa541a753448c8d6d9ab6b68d0d04 P f7cfb1ad38554e1b927a9dad4015e35d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7cfb1ad38554e1b927a9dad4015e35d, State: Initialized, Role: FOLLOWER
I20260807 18:15:08.126436 29278 consensus_queue.cc:260] T 979aa541a753448c8d6d9ab6b68d0d04 P f7cfb1ad38554e1b927a9dad4015e35d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:08.127080 29317 tablet_replica.cc:330] T ecef5162fab040fb89b9a539d58c67c4 P f7cfb1ad38554e1b927a9dad4015e35d: stopping tablet replica
I20260807 18:15:08.127480 29278 ts_tablet_manager.cc:1428] T 979aa541a753448c8d6d9ab6b68d0d04 P f7cfb1ad38554e1b927a9dad4015e35d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 18:15:08.127609 29317 raft_consensus.cc:2229] T ecef5162fab040fb89b9a539d58c67c4 P f7cfb1ad38554e1b927a9dad4015e35d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:15:08.128463 29317 raft_consensus.cc:2258] T ecef5162fab040fb89b9a539d58c67c4 P f7cfb1ad38554e1b927a9dad4015e35d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:15:08.129513 29318 tablet_replica.cc:330] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79: stopping tablet replica
I20260807 18:15:08.130040 29318 raft_consensus.cc:2229] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:15:08.130849 29318 raft_consensus.cc:2258] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:15:08.131079 29277 tablet_bootstrap.cc:492] T 979aa541a753448c8d6d9ab6b68d0d04 P 62030defec5b46a58dd672eeb50f5049: No bootstrap required, opened a new log
I20260807 18:15:08.131313 29277 ts_tablet_manager.cc:1397] T 979aa541a753448c8d6d9ab6b68d0d04 P 62030defec5b46a58dd672eeb50f5049: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.005s
I20260807 18:15:08.132400 29277 raft_consensus.cc:348] T 979aa541a753448c8d6d9ab6b68d0d04 P 62030defec5b46a58dd672eeb50f5049 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:08.133133 29277 raft_consensus.cc:374] T 979aa541a753448c8d6d9ab6b68d0d04 P 62030defec5b46a58dd672eeb50f5049 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:08.133358 29277 raft_consensus.cc:729] T 979aa541a753448c8d6d9ab6b68d0d04 P 62030defec5b46a58dd672eeb50f5049 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62030defec5b46a58dd672eeb50f5049, State: Initialized, Role: FOLLOWER
I20260807 18:15:08.133944 29277 consensus_queue.cc:260] T 979aa541a753448c8d6d9ab6b68d0d04 P 62030defec5b46a58dd672eeb50f5049 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:08.134790 29317 ts_tablet_manager.cc:1905] T ecef5162fab040fb89b9a539d58c67c4 P f7cfb1ad38554e1b927a9dad4015e35d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:15:08.136569 29277 ts_tablet_manager.cc:1428] T 979aa541a753448c8d6d9ab6b68d0d04 P 62030defec5b46a58dd672eeb50f5049: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260807 18:15:08.140487 29318 ts_tablet_manager.cc:1905] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:15:08.145535 29317 ts_tablet_manager.cc:1918] T ecef5162fab040fb89b9a539d58c67c4 P f7cfb1ad38554e1b927a9dad4015e35d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:15:08.145819 29317 log.cc:1196] T ecef5162fab040fb89b9a539d58c67c4 P f7cfb1ad38554e1b927a9dad4015e35d: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-0/wal/wals/ecef5162fab040fb89b9a539d58c67c4
I20260807 18:15:08.146446 29317 ts_tablet_manager.cc:1939] T ecef5162fab040fb89b9a539d58c67c4 P f7cfb1ad38554e1b927a9dad4015e35d: Deleting consensus metadata
I20260807 18:15:08.148664 28554 catalog_manager.cc:4841] TS f7cfb1ad38554e1b927a9dad4015e35d (127.17.110.1:37549): tablet ecef5162fab040fb89b9a539d58c67c4 (table test-table [id=bcda2ad50dbb4ac290284676b1d1f003]) successfully deleted
I20260807 18:15:08.146654 29319 ts_tablet_manager.cc:1918] T ecef5162fab040fb89b9a539d58c67c4 P 62030defec5b46a58dd672eeb50f5049: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:15:08.149917 29319 log.cc:1196] T ecef5162fab040fb89b9a539d58c67c4 P 62030defec5b46a58dd672eeb50f5049: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-2/wal/wals/ecef5162fab040fb89b9a539d58c67c4
I20260807 18:15:08.149927 29318 ts_tablet_manager.cc:1918] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:15:08.150187 29318 log.cc:1196] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-1/wal/wals/ecef5162fab040fb89b9a539d58c67c4
I20260807 18:15:08.150556 29319 ts_tablet_manager.cc:1939] T ecef5162fab040fb89b9a539d58c67c4 P 62030defec5b46a58dd672eeb50f5049: Deleting consensus metadata
I20260807 18:15:08.150725 29318 ts_tablet_manager.cc:1939] T ecef5162fab040fb89b9a539d58c67c4 P bbc6335f70a54d61912f4ed30e2c6f79: Deleting consensus metadata
I20260807 18:15:08.152369 28551 catalog_manager.cc:4841] TS bbc6335f70a54d61912f4ed30e2c6f79 (127.17.110.2:37795): tablet ecef5162fab040fb89b9a539d58c67c4 (table test-table [id=bcda2ad50dbb4ac290284676b1d1f003]) successfully deleted
I20260807 18:15:08.152630 28553 catalog_manager.cc:4841] TS 62030defec5b46a58dd672eeb50f5049 (127.17.110.3:36959): tablet ecef5162fab040fb89b9a539d58c67c4 (table test-table [id=bcda2ad50dbb4ac290284676b1d1f003]) successfully deleted
I20260807 18:15:08.153702 29279 tablet_bootstrap.cc:492] T 979aa541a753448c8d6d9ab6b68d0d04 P bbc6335f70a54d61912f4ed30e2c6f79: No bootstrap required, opened a new log
I20260807 18:15:08.153898 29279 ts_tablet_manager.cc:1397] T 979aa541a753448c8d6d9ab6b68d0d04 P bbc6335f70a54d61912f4ed30e2c6f79: Time spent bootstrapping tablet: real 0.037s user 0.000s sys 0.009s
I20260807 18:15:08.154660 29279 raft_consensus.cc:348] T 979aa541a753448c8d6d9ab6b68d0d04 P bbc6335f70a54d61912f4ed30e2c6f79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:08.154937 29279 raft_consensus.cc:374] T 979aa541a753448c8d6d9ab6b68d0d04 P bbc6335f70a54d61912f4ed30e2c6f79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:08.155048 29279 raft_consensus.cc:729] T 979aa541a753448c8d6d9ab6b68d0d04 P bbc6335f70a54d61912f4ed30e2c6f79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbc6335f70a54d61912f4ed30e2c6f79, State: Initialized, Role: FOLLOWER
I20260807 18:15:08.155361 29279 consensus_queue.cc:260] T 979aa541a753448c8d6d9ab6b68d0d04 P bbc6335f70a54d61912f4ed30e2c6f79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:08.155951 29279 ts_tablet_manager.cc:1428] T 979aa541a753448c8d6d9ab6b68d0d04 P bbc6335f70a54d61912f4ed30e2c6f79: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 18:15:08.171067 29287 raft_consensus.cc:482] T 979aa541a753448c8d6d9ab6b68d0d04 P 62030defec5b46a58dd672eeb50f5049 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:15:08.171360 29287 raft_consensus.cc:504] T 979aa541a753448c8d6d9ab6b68d0d04 P 62030defec5b46a58dd672eeb50f5049 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:08.173396 29287 leader_election.cc:288] T 979aa541a753448c8d6d9ab6b68d0d04 P 62030defec5b46a58dd672eeb50f5049 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7cfb1ad38554e1b927a9dad4015e35d (127.17.110.1:37549), bbc6335f70a54d61912f4ed30e2c6f79 (127.17.110.2:37795)
I20260807 18:15:08.182271 29274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:08.173870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:60695 (local address 127.17.110.2:37795)
0807 18:15:08.174069 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:08.174076 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:08.174775 (+ 699us) server_negotiation.cc:407] Connection header received
0807 18:15:08.175134 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:08.175142 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:08.175257 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:08.175353 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:15:08.176338 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.177585 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.180072 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.181049 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.181259 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:08.181351 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:08.181742 (+ 391us) server_negotiation.cc:299] Negotiation successful
0807 18:15:08.181901 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 18:15:08.182466 29272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:08.174491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:55611 (local address 127.17.110.1:37549)
0807 18:15:08.174759 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:08.174766 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:08.175072 (+ 306us) server_negotiation.cc:407] Connection header received
0807 18:15:08.175351 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:08.175357 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:08.175438 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:08.175537 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:15:08.176411 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.177169 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.180006 (+ 2837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.181383 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.181572 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:08.181665 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:08.182008 (+ 343us) server_negotiation.cc:299] Negotiation successful
0807 18:15:08.182166 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260807 18:15:08.182979 29327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:08.173841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:37549 (local address 127.17.110.3:55611)
0807 18:15:08.174966 (+ 1125us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:08.174986 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:08.175187 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:08.175558 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:08.175561 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:08.175606 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:15:08.176166 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:08.176176 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.177315 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.177323 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:08.179789 (+ 2466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:08.179813 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.181636 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.181645 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:08.181821 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:08.181854 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 18:15:08.182288 (+ 434us) client_negotiation.cc:240] Negotiation successful
0807 18:15:08.182448 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":998,"mutex_wait_us":84,"spinlock_wait_cycles":5888,"thread_start_us":925,"threads_started":1}
I20260807 18:15:08.183073 29326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:08.173677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:37795 (local address 127.17.110.3:60695)
0807 18:15:08.174606 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:08.174630 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:08.174921 (+ 291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:08.175539 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:08.175544 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:08.175611 (+ 67us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:15:08.176139 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:08.176158 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.177767 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.177773 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:08.179913 (+ 2140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:08.179928 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.181290 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.181297 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:08.181502 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:08.181542 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 18:15:08.182497 (+ 955us) client_negotiation.cc:240] Negotiation successful
0807 18:15:08.182653 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":785,"mutex_wait_us":3,"spinlock_wait_cycles":27392,"thread_start_us":612,"threads_started":1}
I20260807 18:15:08.183835 28788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "979aa541a753448c8d6d9ab6b68d0d04" candidate_uuid: "62030defec5b46a58dd672eeb50f5049" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" is_pre_election: true
I20260807 18:15:08.184130 28788 raft_consensus.cc:2454] T 979aa541a753448c8d6d9ab6b68d0d04 P f7cfb1ad38554e1b927a9dad4015e35d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62030defec5b46a58dd672eeb50f5049 in term 0.
I20260807 18:15:08.184262 28976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "979aa541a753448c8d6d9ab6b68d0d04" candidate_uuid: "62030defec5b46a58dd672eeb50f5049" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" is_pre_election: true
I20260807 18:15:08.184756 28976 raft_consensus.cc:2454] T 979aa541a753448c8d6d9ab6b68d0d04 P bbc6335f70a54d61912f4ed30e2c6f79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62030defec5b46a58dd672eeb50f5049 in term 0.
I20260807 18:15:08.184743 29086 leader_election.cc:302] T 979aa541a753448c8d6d9ab6b68d0d04 P 62030defec5b46a58dd672eeb50f5049 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62030defec5b46a58dd672eeb50f5049, f7cfb1ad38554e1b927a9dad4015e35d; no voters:
I20260807 18:15:08.185209 29287 raft_consensus.cc:2781] T 979aa541a753448c8d6d9ab6b68d0d04 P 62030defec5b46a58dd672eeb50f5049 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:15:08.185366 29287 raft_consensus.cc:482] T 979aa541a753448c8d6d9ab6b68d0d04 P 62030defec5b46a58dd672eeb50f5049 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:15:08.185489 29287 raft_consensus.cc:3037] T 979aa541a753448c8d6d9ab6b68d0d04 P 62030defec5b46a58dd672eeb50f5049 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:08.187815 29287 raft_consensus.cc:504] T 979aa541a753448c8d6d9ab6b68d0d04 P 62030defec5b46a58dd672eeb50f5049 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:08.188591 29287 leader_election.cc:288] T 979aa541a753448c8d6d9ab6b68d0d04 P 62030defec5b46a58dd672eeb50f5049 [CANDIDATE]: Term 1 election: Requested vote from peers f7cfb1ad38554e1b927a9dad4015e35d (127.17.110.1:37549), bbc6335f70a54d61912f4ed30e2c6f79 (127.17.110.2:37795)
I20260807 18:15:08.189182 28976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "979aa541a753448c8d6d9ab6b68d0d04" candidate_uuid: "62030defec5b46a58dd672eeb50f5049" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbc6335f70a54d61912f4ed30e2c6f79"
I20260807 18:15:08.189167 28788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "979aa541a753448c8d6d9ab6b68d0d04" candidate_uuid: "62030defec5b46a58dd672eeb50f5049" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7cfb1ad38554e1b927a9dad4015e35d"
I20260807 18:15:08.189409 28788 raft_consensus.cc:3037] T 979aa541a753448c8d6d9ab6b68d0d04 P f7cfb1ad38554e1b927a9dad4015e35d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:08.189416 28976 raft_consensus.cc:3037] T 979aa541a753448c8d6d9ab6b68d0d04 P bbc6335f70a54d61912f4ed30e2c6f79 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:08.192830 28788 raft_consensus.cc:2454] T 979aa541a753448c8d6d9ab6b68d0d04 P f7cfb1ad38554e1b927a9dad4015e35d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62030defec5b46a58dd672eeb50f5049 in term 1.
I20260807 18:15:08.192886 28976 raft_consensus.cc:2454] T 979aa541a753448c8d6d9ab6b68d0d04 P bbc6335f70a54d61912f4ed30e2c6f79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62030defec5b46a58dd672eeb50f5049 in term 1.
I20260807 18:15:08.193490 29081 leader_election.cc:302] T 979aa541a753448c8d6d9ab6b68d0d04 P 62030defec5b46a58dd672eeb50f5049 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62030defec5b46a58dd672eeb50f5049, bbc6335f70a54d61912f4ed30e2c6f79; no voters:
I20260807 18:15:08.193959 29287 raft_consensus.cc:2781] T 979aa541a753448c8d6d9ab6b68d0d04 P 62030defec5b46a58dd672eeb50f5049 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:15:08.194466 29287 raft_consensus.cc:686] T 979aa541a753448c8d6d9ab6b68d0d04 P 62030defec5b46a58dd672eeb50f5049 [term 1 LEADER]: Becoming Leader. State: Replica: 62030defec5b46a58dd672eeb50f5049, State: Running, Role: LEADER
I20260807 18:15:08.194979 29287 consensus_queue.cc:237] T 979aa541a753448c8d6d9ab6b68d0d04 P 62030defec5b46a58dd672eeb50f5049 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:08.199362 28570 catalog_manager.cc:5495] T 979aa541a753448c8d6d9ab6b68d0d04 P 62030defec5b46a58dd672eeb50f5049 reported cstate change: term changed from 0 to 1, leader changed from <none> to 62030defec5b46a58dd672eeb50f5049 (127.17.110.3). New cstate: current_term: 1 leader_uuid: "62030defec5b46a58dd672eeb50f5049" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } health_report { overall_health: HEALTHY } } }
Aug 07 18:15:08 dist-test-slave-39l8 krb5kdc[28513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126504, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.62@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM
I20260807 18:15:08.275498 29336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:08.257758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40437 (local address 127.0.0.1:50790)
0807 18:15:08.258831 (+ 1073us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:08.258858 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:08.259034 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:08.259466 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:08.259472 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:08.260423 (+ 951us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:08.260736 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:08.260752 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.262041 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.262048 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:08.263724 (+ 1676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:08.263743 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.264601 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.264608 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:08.264821 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:08.265909 (+ 1088us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:08.265944 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:08.268988 (+ 3044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:08.272562 (+ 3574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:08.272582 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:08.272591 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:08.273093 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:08.273485 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:08.273492 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:08.273495 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:08.273639 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:08.274214 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:08.274224 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:08.274410 (+ 186us) client_negotiation.cc:769] Sending connection context
0807 18:15:08.274579 (+ 169us) client_negotiation.cc:240] Negotiation successful
0807 18:15:08.274947 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":793,"thread_start_us":601,"threads_started":1}
I20260807 18:15:08.275520 29335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:08.257759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50790 (local address 127.17.110.62:40437)
0807 18:15:08.258593 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:08.258600 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:08.258956 (+ 356us) server_negotiation.cc:407] Connection header received
0807 18:15:08.259156 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:08.259160 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:08.259223 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:08.259386 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:08.261000 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.261874 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.263912 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.264397 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.264516 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:08.269328 (+ 4812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:08.269333 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:08.269337 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:08.269373 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:08.272387 (+ 3014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:08.273233 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:08.273238 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:08.273246 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:08.273369 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:08.273755 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:08.273760 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:08.273763 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:08.274076 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:08.274240 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:08.274689 (+ 449us) server_negotiation.cc:299] Negotiation successful
0807 18:15:08.275096 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":151,"server-negotiator.queue_time_us":625,"thread_start_us":375,"threads_started":1}
I20260807 18:15:08.281813 28570 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:50790:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 18:15:08.283191 28570 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:15:08.291543 28758 tablet_service.cc:1467] Processing CreateTablet for tablet 68cf0fbc22844dff85b808d582e57d88 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b052b4e294244f41a41d7691c7bf15ac]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:15:08.292224 28758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68cf0fbc22844dff85b808d582e57d88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:08.292472 28945 tablet_service.cc:1467] Processing CreateTablet for tablet 68cf0fbc22844dff85b808d582e57d88 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b052b4e294244f41a41d7691c7bf15ac]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:15:08.293102 28945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68cf0fbc22844dff85b808d582e57d88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:08.298074 29146 tablet_service.cc:1467] Processing CreateTablet for tablet 68cf0fbc22844dff85b808d582e57d88 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b052b4e294244f41a41d7691c7bf15ac]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:15:08.299362 29146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68cf0fbc22844dff85b808d582e57d88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:08.299868 29278 tablet_bootstrap.cc:492] T 68cf0fbc22844dff85b808d582e57d88 P f7cfb1ad38554e1b927a9dad4015e35d: Bootstrap starting.
I20260807 18:15:08.301910 29278 tablet_bootstrap.cc:654] T 68cf0fbc22844dff85b808d582e57d88 P f7cfb1ad38554e1b927a9dad4015e35d: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:08.305464 29278 tablet_bootstrap.cc:492] T 68cf0fbc22844dff85b808d582e57d88 P f7cfb1ad38554e1b927a9dad4015e35d: No bootstrap required, opened a new log
I20260807 18:15:08.305671 29278 ts_tablet_manager.cc:1397] T 68cf0fbc22844dff85b808d582e57d88 P f7cfb1ad38554e1b927a9dad4015e35d: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260807 18:15:08.306471 29278 raft_consensus.cc:348] T 68cf0fbc22844dff85b808d582e57d88 P f7cfb1ad38554e1b927a9dad4015e35d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:08.306771 29278 raft_consensus.cc:374] T 68cf0fbc22844dff85b808d582e57d88 P f7cfb1ad38554e1b927a9dad4015e35d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:08.306856 29278 raft_consensus.cc:729] T 68cf0fbc22844dff85b808d582e57d88 P f7cfb1ad38554e1b927a9dad4015e35d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7cfb1ad38554e1b927a9dad4015e35d, State: Initialized, Role: FOLLOWER
I20260807 18:15:08.307148 29278 consensus_queue.cc:260] T 68cf0fbc22844dff85b808d582e57d88 P f7cfb1ad38554e1b927a9dad4015e35d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:08.307657 29284 tablet_replica.cc:439] T 68cf0fbc22844dff85b808d582e57d88 P f7cfb1ad38554e1b927a9dad4015e35d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } } }
I20260807 18:15:08.308090 29284 tablet_replica.cc:442] T 68cf0fbc22844dff85b808d582e57d88 P f7cfb1ad38554e1b927a9dad4015e35d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:15:08.308583 29278 ts_tablet_manager.cc:1428] T 68cf0fbc22844dff85b808d582e57d88 P f7cfb1ad38554e1b927a9dad4015e35d: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 18:15:08.311352 29279 tablet_bootstrap.cc:492] T 68cf0fbc22844dff85b808d582e57d88 P bbc6335f70a54d61912f4ed30e2c6f79: Bootstrap starting.
I20260807 18:15:08.313261 29279 tablet_bootstrap.cc:654] T 68cf0fbc22844dff85b808d582e57d88 P bbc6335f70a54d61912f4ed30e2c6f79: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:08.314877 29277 tablet_bootstrap.cc:492] T 68cf0fbc22844dff85b808d582e57d88 P 62030defec5b46a58dd672eeb50f5049: Bootstrap starting.
I20260807 18:15:08.316419 29279 tablet_bootstrap.cc:492] T 68cf0fbc22844dff85b808d582e57d88 P bbc6335f70a54d61912f4ed30e2c6f79: No bootstrap required, opened a new log
I20260807 18:15:08.316632 29279 ts_tablet_manager.cc:1397] T 68cf0fbc22844dff85b808d582e57d88 P bbc6335f70a54d61912f4ed30e2c6f79: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20260807 18:15:08.317005 29277 tablet_bootstrap.cc:654] T 68cf0fbc22844dff85b808d582e57d88 P 62030defec5b46a58dd672eeb50f5049: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:08.317484 29279 raft_consensus.cc:348] T 68cf0fbc22844dff85b808d582e57d88 P bbc6335f70a54d61912f4ed30e2c6f79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:08.317777 29279 raft_consensus.cc:374] T 68cf0fbc22844dff85b808d582e57d88 P bbc6335f70a54d61912f4ed30e2c6f79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:08.317874 29279 raft_consensus.cc:729] T 68cf0fbc22844dff85b808d582e57d88 P bbc6335f70a54d61912f4ed30e2c6f79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbc6335f70a54d61912f4ed30e2c6f79, State: Initialized, Role: FOLLOWER
I20260807 18:15:08.318217 29279 consensus_queue.cc:260] T 68cf0fbc22844dff85b808d582e57d88 P bbc6335f70a54d61912f4ed30e2c6f79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:08.318778 29311 tablet_replica.cc:439] T 68cf0fbc22844dff85b808d582e57d88 P bbc6335f70a54d61912f4ed30e2c6f79: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } } }
I20260807 18:15:08.319257 29311 tablet_replica.cc:442] T 68cf0fbc22844dff85b808d582e57d88 P bbc6335f70a54d61912f4ed30e2c6f79: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:15:08.320356 29279 ts_tablet_manager.cc:1428] T 68cf0fbc22844dff85b808d582e57d88 P bbc6335f70a54d61912f4ed30e2c6f79: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260807 18:15:08.323712 29277 tablet_bootstrap.cc:492] T 68cf0fbc22844dff85b808d582e57d88 P 62030defec5b46a58dd672eeb50f5049: No bootstrap required, opened a new log
I20260807 18:15:08.323933 29277 ts_tablet_manager.cc:1397] T 68cf0fbc22844dff85b808d582e57d88 P 62030defec5b46a58dd672eeb50f5049: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260807 18:15:08.324823 29277 raft_consensus.cc:348] T 68cf0fbc22844dff85b808d582e57d88 P 62030defec5b46a58dd672eeb50f5049 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:08.325069 29277 raft_consensus.cc:374] T 68cf0fbc22844dff85b808d582e57d88 P 62030defec5b46a58dd672eeb50f5049 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:08.325166 29277 raft_consensus.cc:729] T 68cf0fbc22844dff85b808d582e57d88 P 62030defec5b46a58dd672eeb50f5049 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62030defec5b46a58dd672eeb50f5049, State: Initialized, Role: FOLLOWER
I20260807 18:15:08.325469 29277 consensus_queue.cc:260] T 68cf0fbc22844dff85b808d582e57d88 P 62030defec5b46a58dd672eeb50f5049 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:08.325910 29287 tablet_replica.cc:439] T 68cf0fbc22844dff85b808d582e57d88 P 62030defec5b46a58dd672eeb50f5049: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } } }
I20260807 18:15:08.326135 29277 ts_tablet_manager.cc:1428] T 68cf0fbc22844dff85b808d582e57d88 P 62030defec5b46a58dd672eeb50f5049: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260807 18:15:08.326267 29287 tablet_replica.cc:442] T 68cf0fbc22844dff85b808d582e57d88 P 62030defec5b46a58dd672eeb50f5049: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:15:08.420686 29311 raft_consensus.cc:482] T 68cf0fbc22844dff85b808d582e57d88 P bbc6335f70a54d61912f4ed30e2c6f79 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:15:08.420959 29311 raft_consensus.cc:504] T 68cf0fbc22844dff85b808d582e57d88 P bbc6335f70a54d61912f4ed30e2c6f79 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:08.421902 29311 leader_election.cc:288] T 68cf0fbc22844dff85b808d582e57d88 P bbc6335f70a54d61912f4ed30e2c6f79 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7cfb1ad38554e1b927a9dad4015e35d (127.17.110.1:37549), 62030defec5b46a58dd672eeb50f5049 (127.17.110.3:36959)
I20260807 18:15:08.422643 28788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68cf0fbc22844dff85b808d582e57d88" candidate_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" is_pre_election: true
I20260807 18:15:08.422749 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68cf0fbc22844dff85b808d582e57d88" candidate_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62030defec5b46a58dd672eeb50f5049" is_pre_election: true
I20260807 18:15:08.423046 28788 raft_consensus.cc:2454] T 68cf0fbc22844dff85b808d582e57d88 P f7cfb1ad38554e1b927a9dad4015e35d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbc6335f70a54d61912f4ed30e2c6f79 in term 0.
I20260807 18:15:08.423067 29175 raft_consensus.cc:2454] T 68cf0fbc22844dff85b808d582e57d88 P 62030defec5b46a58dd672eeb50f5049 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbc6335f70a54d61912f4ed30e2c6f79 in term 0.
I20260807 18:15:08.423595 28885 leader_election.cc:302] T 68cf0fbc22844dff85b808d582e57d88 P bbc6335f70a54d61912f4ed30e2c6f79 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbc6335f70a54d61912f4ed30e2c6f79, f7cfb1ad38554e1b927a9dad4015e35d; no voters:
I20260807 18:15:08.423936 29311 raft_consensus.cc:2781] T 68cf0fbc22844dff85b808d582e57d88 P bbc6335f70a54d61912f4ed30e2c6f79 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:15:08.424108 29311 raft_consensus.cc:482] T 68cf0fbc22844dff85b808d582e57d88 P bbc6335f70a54d61912f4ed30e2c6f79 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:15:08.424230 29311 raft_consensus.cc:3037] T 68cf0fbc22844dff85b808d582e57d88 P bbc6335f70a54d61912f4ed30e2c6f79 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:08.427313 29311 raft_consensus.cc:504] T 68cf0fbc22844dff85b808d582e57d88 P bbc6335f70a54d61912f4ed30e2c6f79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:08.428181 29311 leader_election.cc:288] T 68cf0fbc22844dff85b808d582e57d88 P bbc6335f70a54d61912f4ed30e2c6f79 [CANDIDATE]: Term 1 election: Requested vote from peers f7cfb1ad38554e1b927a9dad4015e35d (127.17.110.1:37549), 62030defec5b46a58dd672eeb50f5049 (127.17.110.3:36959)
I20260807 18:15:08.428793 28788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68cf0fbc22844dff85b808d582e57d88" candidate_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7cfb1ad38554e1b927a9dad4015e35d"
I20260807 18:15:08.428905 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68cf0fbc22844dff85b808d582e57d88" candidate_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62030defec5b46a58dd672eeb50f5049"
I20260807 18:15:08.429095 28788 raft_consensus.cc:3037] T 68cf0fbc22844dff85b808d582e57d88 P f7cfb1ad38554e1b927a9dad4015e35d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:08.429183 29175 raft_consensus.cc:3037] T 68cf0fbc22844dff85b808d582e57d88 P 62030defec5b46a58dd672eeb50f5049 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:08.431860 29175 raft_consensus.cc:2454] T 68cf0fbc22844dff85b808d582e57d88 P 62030defec5b46a58dd672eeb50f5049 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbc6335f70a54d61912f4ed30e2c6f79 in term 1.
I20260807 18:15:08.432471 28788 raft_consensus.cc:2454] T 68cf0fbc22844dff85b808d582e57d88 P f7cfb1ad38554e1b927a9dad4015e35d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbc6335f70a54d61912f4ed30e2c6f79 in term 1.
I20260807 18:15:08.432576 28884 leader_election.cc:302] T 68cf0fbc22844dff85b808d582e57d88 P bbc6335f70a54d61912f4ed30e2c6f79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62030defec5b46a58dd672eeb50f5049, bbc6335f70a54d61912f4ed30e2c6f79; no voters:
I20260807 18:15:08.433168 29311 raft_consensus.cc:2781] T 68cf0fbc22844dff85b808d582e57d88 P bbc6335f70a54d61912f4ed30e2c6f79 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:15:08.433487 29311 raft_consensus.cc:686] T 68cf0fbc22844dff85b808d582e57d88 P bbc6335f70a54d61912f4ed30e2c6f79 [term 1 LEADER]: Becoming Leader. State: Replica: bbc6335f70a54d61912f4ed30e2c6f79, State: Running, Role: LEADER
I20260807 18:15:08.433897 29311 consensus_queue.cc:237] T 68cf0fbc22844dff85b808d582e57d88 P bbc6335f70a54d61912f4ed30e2c6f79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:08.436167 29288 tablet_replica.cc:439] T 68cf0fbc22844dff85b808d582e57d88 P bbc6335f70a54d61912f4ed30e2c6f79: TxnStatusTablet state changed. Reason: New leader bbc6335f70a54d61912f4ed30e2c6f79. Latest consensus state: current_term: 1 leader_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } } }
I20260807 18:15:08.436506 29288 tablet_replica.cc:442] T 68cf0fbc22844dff85b808d582e57d88 P bbc6335f70a54d61912f4ed30e2c6f79: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:15:08.437521 29343 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 18:15:08.437834 29343 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 18:15:08.438941 28570 catalog_manager.cc:5495] T 68cf0fbc22844dff85b808d582e57d88 P bbc6335f70a54d61912f4ed30e2c6f79 reported cstate change: term changed from 0 to 1, leader changed from <none> to bbc6335f70a54d61912f4ed30e2c6f79 (127.17.110.2). New cstate: current_term: 1 leader_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } health_report { overall_health: UNKNOWN } } }
I20260807 18:15:08.479213 29274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:08.473103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40044 (local address 127.17.110.2:37795)
0807 18:15:08.473342 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:08.473350 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:08.473381 (+ 31us) server_negotiation.cc:407] Connection header received
0807 18:15:08.473589 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:08.473599 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:08.473694 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:08.473806 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:15:08.474199 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.475466 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.476908 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.477595 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.477768 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:08.478292 (+ 524us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:15:08.478450 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:15:08.478570 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:08.478788 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 18:15:08.478886 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260807 18:15:08.479251 29336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:08.472952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:37795 (local address 127.0.0.1:40044)
0807 18:15:08.473233 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:08.473263 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:08.473392 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:08.473858 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:08.473863 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:08.473874 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:15:08.474038 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:08.474046 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.475665 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.475676 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:08.476725 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:08.476740 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.477818 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.477823 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:08.477995 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:08.478067 (+ 72us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:15:08.478588 (+ 521us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:15:08.478615 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 18:15:08.478856 (+ 241us) client_negotiation.cc:240] Negotiation successful
0807 18:15:08.478997 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 18:15:08.483608 28788 raft_consensus.cc:1261] T 68cf0fbc22844dff85b808d582e57d88 P f7cfb1ad38554e1b927a9dad4015e35d [term 1 FOLLOWER]: Refusing update from remote peer bbc6335f70a54d61912f4ed30e2c6f79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:15:08.483665 29175 raft_consensus.cc:1261] T 68cf0fbc22844dff85b808d582e57d88 P 62030defec5b46a58dd672eeb50f5049 [term 1 FOLLOWER]: Refusing update from remote peer bbc6335f70a54d61912f4ed30e2c6f79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:15:08.484369 29311 consensus_queue.cc:1035] T 68cf0fbc22844dff85b808d582e57d88 P bbc6335f70a54d61912f4ed30e2c6f79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:15:08.484692 29288 consensus_queue.cc:1035] T 68cf0fbc22844dff85b808d582e57d88 P bbc6335f70a54d61912f4ed30e2c6f79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:15:08.487920 29287 tablet_replica.cc:439] T 68cf0fbc22844dff85b808d582e57d88 P 62030defec5b46a58dd672eeb50f5049: TxnStatusTablet state changed. Reason: New leader bbc6335f70a54d61912f4ed30e2c6f79. Latest consensus state: current_term: 1 leader_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } } }
I20260807 18:15:08.488265 29287 tablet_replica.cc:442] T 68cf0fbc22844dff85b808d582e57d88 P 62030defec5b46a58dd672eeb50f5049: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:15:08.491518 29311 tablet_replica.cc:439] T 68cf0fbc22844dff85b808d582e57d88 P bbc6335f70a54d61912f4ed30e2c6f79: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } } }
I20260807 18:15:08.491897 29311 tablet_replica.cc:442] T 68cf0fbc22844dff85b808d582e57d88 P bbc6335f70a54d61912f4ed30e2c6f79: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:15:08.496057 29302 mvcc.cc:205] Tried to move back new op lower bound from 7315974178741284864 to 7315974178548908032. Current Snapshot: MvccSnapshot[applied={T|T < 7315974178741284864 or (T in {7315974178741284864})}]
I20260807 18:15:08.496508 29287 tablet_replica.cc:439] T 68cf0fbc22844dff85b808d582e57d88 P 62030defec5b46a58dd672eeb50f5049: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } } }
I20260807 18:15:08.496846 29287 tablet_replica.cc:442] T 68cf0fbc22844dff85b808d582e57d88 P 62030defec5b46a58dd672eeb50f5049: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:15:08.500600 29288 tablet_replica.cc:439] T 68cf0fbc22844dff85b808d582e57d88 P bbc6335f70a54d61912f4ed30e2c6f79: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } } }
I20260807 18:15:08.500943 29288 tablet_replica.cc:442] T 68cf0fbc22844dff85b808d582e57d88 P bbc6335f70a54d61912f4ed30e2c6f79: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:15:08.501811 29284 tablet_replica.cc:439] T 68cf0fbc22844dff85b808d582e57d88 P f7cfb1ad38554e1b927a9dad4015e35d: TxnStatusTablet state changed. Reason: New leader bbc6335f70a54d61912f4ed30e2c6f79. Latest consensus state: current_term: 1 leader_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } } }
I20260807 18:15:08.502149 29284 tablet_replica.cc:442] T 68cf0fbc22844dff85b808d582e57d88 P f7cfb1ad38554e1b927a9dad4015e35d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:15:08.505448 29284 tablet_replica.cc:439] T 68cf0fbc22844dff85b808d582e57d88 P f7cfb1ad38554e1b927a9dad4015e35d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } } }
I20260807 18:15:08.505728 29284 tablet_replica.cc:442] T 68cf0fbc22844dff85b808d582e57d88 P f7cfb1ad38554e1b927a9dad4015e35d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:15:08.508834 29348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:08.499988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48258 (local address 127.17.110.3:36959)
0807 18:15:08.500556 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:08.500562 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:08.500588 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:15:08.500697 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:08.500702 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:08.500821 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:08.501691 (+ 870us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:15:08.501875 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.503205 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.505225 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.506103 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.506258 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:08.506864 (+ 606us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:15:08.507649 (+ 785us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:15:08.508314 (+ 665us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:08.508375 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 18:15:08.508475 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":2802,"threads_started":1}
I20260807 18:15:08.528239 29274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:08.520979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:57163 (local address 127.17.110.2:37795)
0807 18:15:08.521226 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:08.521233 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:08.521700 (+ 467us) server_negotiation.cc:407] Connection header received
0807 18:15:08.521886 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:08.521893 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:08.521999 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:08.522121 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:15:08.522658 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.523930 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.525685 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.526476 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.526611 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:08.527198 (+ 587us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:15:08.527353 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:15:08.527449 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:08.527652 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 18:15:08.527760 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:15:08.528556 29355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:08.520883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:37795 (local address 127.17.110.3:57163)
0807 18:15:08.521581 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:08.521599 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:08.521737 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:08.522151 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:08.522155 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:08.522165 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:15:08.522469 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:08.522477 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.524101 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.524107 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:08.525505 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:08.525522 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.526665 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.526671 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:08.526885 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:08.526998 (+ 113us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:15:08.527507 (+ 509us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:15:08.527521 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 18:15:08.527614 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 18:15:08.527749 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":394,"threads_started":1}
I20260807 18:15:08.540725 28976 raft_consensus.cc:1261] T 979aa541a753448c8d6d9ab6b68d0d04 P bbc6335f70a54d61912f4ed30e2c6f79 [term 1 FOLLOWER]: Refusing update from remote peer 62030defec5b46a58dd672eeb50f5049: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:15:08.540694 28788 raft_consensus.cc:1261] T 979aa541a753448c8d6d9ab6b68d0d04 P f7cfb1ad38554e1b927a9dad4015e35d [term 1 FOLLOWER]: Refusing update from remote peer 62030defec5b46a58dd672eeb50f5049: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:15:08.541567 29328 consensus_queue.cc:1035] T 979aa541a753448c8d6d9ab6b68d0d04 P 62030defec5b46a58dd672eeb50f5049 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:15:08.542400 29328 consensus_queue.cc:1035] T 979aa541a753448c8d6d9ab6b68d0d04 P 62030defec5b46a58dd672eeb50f5049 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:15:08.583151 29348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:08.575945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:42783 (local address 127.17.110.3:36959)
0807 18:15:08.576137 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:08.576144 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:08.576416 (+ 272us) server_negotiation.cc:407] Connection header received
0807 18:15:08.576770 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:08.576776 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:08.576866 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:08.576978 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:15:08.577759 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.578533 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.580978 (+ 2445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.581620 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.581773 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:08.582416 (+ 643us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:15:08.582533 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:15:08.582610 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:08.582841 (+ 231us) server_negotiation.cc:299] Negotiation successful
0807 18:15:08.582914 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 18:15:08.583566 29362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:08.575343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:36959 (local address 127.17.110.2:42783)
0807 18:15:08.576282 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:08.576312 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:08.576591 (+ 279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:08.577015 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:08.577023 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:08.577038 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:15:08.577519 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:08.577533 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.578701 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.578736 (+ 35us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:08.580771 (+ 2035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:08.580793 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.581841 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.581849 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:08.582077 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:08.582192 (+ 115us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:15:08.582653 (+ 461us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:15:08.582668 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 18:15:08.582821 (+ 153us) client_negotiation.cc:240] Negotiation successful
0807 18:15:08.583013 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":815,"threads_started":1}
I20260807 18:15:08.650031 28570 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:15:08.650362 28570 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:15:08.655349 28570 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f89b8e74b0ae4f2daab1653a7fbb4e00: Sending DeleteTablet for 3 replicas of tablet 979aa541a753448c8d6d9ab6b68d0d04
I20260807 18:15:08.656766 28758 tablet_service.cc:1514] Processing DeleteTablet for tablet 979aa541a753448c8d6d9ab6b68d0d04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:15:08 UTC) from {username='oryx', principal='oryx/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:56400
I20260807 18:15:08.657855 29146 tablet_service.cc:1514] Processing DeleteTablet for tablet 979aa541a753448c8d6d9ab6b68d0d04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:15:08 UTC) from {username='oryx', principal='oryx/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:48250
I20260807 18:15:08.659246 28945 tablet_service.cc:1514] Processing DeleteTablet for tablet 979aa541a753448c8d6d9ab6b68d0d04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:15:08 UTC) from {username='oryx', principal='oryx/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:40028
I20260807 18:15:08.667220 29372 tablet_replica.cc:330] T 979aa541a753448c8d6d9ab6b68d0d04 P 62030defec5b46a58dd672eeb50f5049: stopping tablet replica
I20260807 18:15:08.667500 29372 raft_consensus.cc:2229] T 979aa541a753448c8d6d9ab6b68d0d04 P 62030defec5b46a58dd672eeb50f5049 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:15:08.668231 29372 raft_consensus.cc:2258] T 979aa541a753448c8d6d9ab6b68d0d04 P 62030defec5b46a58dd672eeb50f5049 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:15:08.669664 29372 ts_tablet_manager.cc:1905] T 979aa541a753448c8d6d9ab6b68d0d04 P 62030defec5b46a58dd672eeb50f5049: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:15:08.678244 29372 ts_tablet_manager.cc:1918] T 979aa541a753448c8d6d9ab6b68d0d04 P 62030defec5b46a58dd672eeb50f5049: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:15:08.678444 29372 log.cc:1196] T 979aa541a753448c8d6d9ab6b68d0d04 P 62030defec5b46a58dd672eeb50f5049: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-2/wal/wals/979aa541a753448c8d6d9ab6b68d0d04
I20260807 18:15:08.680172 29372 ts_tablet_manager.cc:1939] T 979aa541a753448c8d6d9ab6b68d0d04 P 62030defec5b46a58dd672eeb50f5049: Deleting consensus metadata
I20260807 18:15:08.681643 28553 catalog_manager.cc:4841] TS 62030defec5b46a58dd672eeb50f5049 (127.17.110.3:36959): tablet 979aa541a753448c8d6d9ab6b68d0d04 (table test-table [id=72845f2d237c4102a39d1d78e5ad3e55]) successfully deleted
I20260807 18:15:08.686311 29373 tablet_replica.cc:330] T 979aa541a753448c8d6d9ab6b68d0d04 P bbc6335f70a54d61912f4ed30e2c6f79: stopping tablet replica
I20260807 18:15:08.686581 29373 raft_consensus.cc:2229] T 979aa541a753448c8d6d9ab6b68d0d04 P bbc6335f70a54d61912f4ed30e2c6f79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:15:08.686903 29373 raft_consensus.cc:2258] T 979aa541a753448c8d6d9ab6b68d0d04 P bbc6335f70a54d61912f4ed30e2c6f79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:15:08.688272 29373 ts_tablet_manager.cc:1905] T 979aa541a753448c8d6d9ab6b68d0d04 P bbc6335f70a54d61912f4ed30e2c6f79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:15:08.691247 29370 tablet_replica.cc:330] T 979aa541a753448c8d6d9ab6b68d0d04 P f7cfb1ad38554e1b927a9dad4015e35d: stopping tablet replica
I20260807 18:15:08.691488 29370 raft_consensus.cc:2229] T 979aa541a753448c8d6d9ab6b68d0d04 P f7cfb1ad38554e1b927a9dad4015e35d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:15:08.691756 29370 raft_consensus.cc:2258] T 979aa541a753448c8d6d9ab6b68d0d04 P f7cfb1ad38554e1b927a9dad4015e35d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:15:08.693277 29370 ts_tablet_manager.cc:1905] T 979aa541a753448c8d6d9ab6b68d0d04 P f7cfb1ad38554e1b927a9dad4015e35d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:15:08.697832 29373 ts_tablet_manager.cc:1918] T 979aa541a753448c8d6d9ab6b68d0d04 P bbc6335f70a54d61912f4ed30e2c6f79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:15:08.698025 29373 log.cc:1196] T 979aa541a753448c8d6d9ab6b68d0d04 P bbc6335f70a54d61912f4ed30e2c6f79: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-1/wal/wals/979aa541a753448c8d6d9ab6b68d0d04
I20260807 18:15:08.699105 29373 ts_tablet_manager.cc:1939] T 979aa541a753448c8d6d9ab6b68d0d04 P bbc6335f70a54d61912f4ed30e2c6f79: Deleting consensus metadata
I20260807 18:15:08.700999 28551 catalog_manager.cc:4841] TS bbc6335f70a54d61912f4ed30e2c6f79 (127.17.110.2:37795): tablet 979aa541a753448c8d6d9ab6b68d0d04 (table test-table [id=72845f2d237c4102a39d1d78e5ad3e55]) successfully deleted
I20260807 18:15:08.704964 29370 ts_tablet_manager.cc:1918] T 979aa541a753448c8d6d9ab6b68d0d04 P f7cfb1ad38554e1b927a9dad4015e35d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:15:08.705169 29370 log.cc:1196] T 979aa541a753448c8d6d9ab6b68d0d04 P f7cfb1ad38554e1b927a9dad4015e35d: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-0/wal/wals/979aa541a753448c8d6d9ab6b68d0d04
I20260807 18:15:08.705648 29370 ts_tablet_manager.cc:1939] T 979aa541a753448c8d6d9ab6b68d0d04 P f7cfb1ad38554e1b927a9dad4015e35d: Deleting consensus metadata
I20260807 18:15:08.707103 28554 catalog_manager.cc:4841] TS f7cfb1ad38554e1b927a9dad4015e35d (127.17.110.1:37549): tablet 979aa541a753448c8d6d9ab6b68d0d04 (table test-table [id=72845f2d237c4102a39d1d78e5ad3e55]) successfully deleted
Aug 07 18:15:08 dist-test-slave-39l8 krb5kdc[28513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 18:15:08 dist-test-slave-39l8 krb5kdc[28513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:08 dist-test-slave-39l8 krb5kdc[28513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 18:15:08.778501 29335 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:08.762412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50796 (local address 127.17.110.62:40437)
0807 18:15:08.762663 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:08.762671 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:08.763310 (+ 639us) server_negotiation.cc:407] Connection header received
0807 18:15:08.763519 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:08.763525 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:08.763627 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:08.763754 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:08.765905 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.767311 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.769404 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.770197 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.770352 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:08.778184 (+ 7832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50796: BlockingRecv error: recv got EOF from 127.0.0.1:50796 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 07 18:15:08 dist-test-slave-39l8 krb5kdc[28513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM
I20260807 18:15:08.807868 29335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:08.784836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50802 (local address 127.17.110.62:40437)
0807 18:15:08.785059 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:08.785067 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:08.785474 (+ 407us) server_negotiation.cc:407] Connection header received
0807 18:15:08.785813 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:08.785820 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:08.785917 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:08.786011 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:08.788200 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.789581 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.792135 (+ 2554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.792939 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.793073 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:08.799500 (+ 6427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:08.799509 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:08.799514 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:08.799596 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:08.803996 (+ 4400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:08.805186 (+ 1190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:08.805194 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:08.805198 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:08.805412 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:08.805932 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:08.805939 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:08.805943 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:08.806355 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:08.806687 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:08.807089 (+ 402us) server_negotiation.cc:299] Negotiation successful
0807 18:15:08.807406 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 18:15:08.812994 28570 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50802:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:15:08.813810 28570 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:15:08.824234 28945 tablet_service.cc:1467] Processing CreateTablet for tablet 3bceca7cc97447b8bbf4b7d07eb18598 (DEFAULT_TABLE table=test-table [id=5423c742e4884a46a0859cf0e6e21297]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:15:08.824885 28945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bceca7cc97447b8bbf4b7d07eb18598. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:08.827463 28758 tablet_service.cc:1467] Processing CreateTablet for tablet 3bceca7cc97447b8bbf4b7d07eb18598 (DEFAULT_TABLE table=test-table [id=5423c742e4884a46a0859cf0e6e21297]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:15:08.827991 28758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bceca7cc97447b8bbf4b7d07eb18598. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:08.831214 29146 tablet_service.cc:1467] Processing CreateTablet for tablet 3bceca7cc97447b8bbf4b7d07eb18598 (DEFAULT_TABLE table=test-table [id=5423c742e4884a46a0859cf0e6e21297]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:15:08.831735 29146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bceca7cc97447b8bbf4b7d07eb18598. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:08.844970 29393 tablet_bootstrap.cc:492] T 3bceca7cc97447b8bbf4b7d07eb18598 P bbc6335f70a54d61912f4ed30e2c6f79: Bootstrap starting.
I20260807 18:15:08.848588 29393 tablet_bootstrap.cc:654] T 3bceca7cc97447b8bbf4b7d07eb18598 P bbc6335f70a54d61912f4ed30e2c6f79: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:08.859898 29398 tablet_bootstrap.cc:492] T 3bceca7cc97447b8bbf4b7d07eb18598 P f7cfb1ad38554e1b927a9dad4015e35d: Bootstrap starting.
I20260807 18:15:08.861387 29393 tablet_bootstrap.cc:492] T 3bceca7cc97447b8bbf4b7d07eb18598 P bbc6335f70a54d61912f4ed30e2c6f79: No bootstrap required, opened a new log
I20260807 18:15:08.861594 29393 ts_tablet_manager.cc:1397] T 3bceca7cc97447b8bbf4b7d07eb18598 P bbc6335f70a54d61912f4ed30e2c6f79: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.008s
I20260807 18:15:08.862534 29393 raft_consensus.cc:348] T 3bceca7cc97447b8bbf4b7d07eb18598 P bbc6335f70a54d61912f4ed30e2c6f79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:08.862879 29393 raft_consensus.cc:374] T 3bceca7cc97447b8bbf4b7d07eb18598 P bbc6335f70a54d61912f4ed30e2c6f79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:08.862993 29393 raft_consensus.cc:729] T 3bceca7cc97447b8bbf4b7d07eb18598 P bbc6335f70a54d61912f4ed30e2c6f79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbc6335f70a54d61912f4ed30e2c6f79, State: Initialized, Role: FOLLOWER
I20260807 18:15:08.863587 29394 tablet_bootstrap.cc:492] T 3bceca7cc97447b8bbf4b7d07eb18598 P 62030defec5b46a58dd672eeb50f5049: Bootstrap starting.
I20260807 18:15:08.864903 29393 consensus_queue.cc:260] T 3bceca7cc97447b8bbf4b7d07eb18598 P bbc6335f70a54d61912f4ed30e2c6f79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:08.865845 29398 tablet_bootstrap.cc:654] T 3bceca7cc97447b8bbf4b7d07eb18598 P f7cfb1ad38554e1b927a9dad4015e35d: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:08.868259 29394 tablet_bootstrap.cc:654] T 3bceca7cc97447b8bbf4b7d07eb18598 P 62030defec5b46a58dd672eeb50f5049: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:08.869210 29393 ts_tablet_manager.cc:1428] T 3bceca7cc97447b8bbf4b7d07eb18598 P bbc6335f70a54d61912f4ed30e2c6f79: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260807 18:15:08.878521 29398 tablet_bootstrap.cc:492] T 3bceca7cc97447b8bbf4b7d07eb18598 P f7cfb1ad38554e1b927a9dad4015e35d: No bootstrap required, opened a new log
I20260807 18:15:08.878769 29398 ts_tablet_manager.cc:1397] T 3bceca7cc97447b8bbf4b7d07eb18598 P f7cfb1ad38554e1b927a9dad4015e35d: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.004s
I20260807 18:15:08.879170 29394 tablet_bootstrap.cc:492] T 3bceca7cc97447b8bbf4b7d07eb18598 P 62030defec5b46a58dd672eeb50f5049: No bootstrap required, opened a new log
I20260807 18:15:08.879472 29394 ts_tablet_manager.cc:1397] T 3bceca7cc97447b8bbf4b7d07eb18598 P 62030defec5b46a58dd672eeb50f5049: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.004s
I20260807 18:15:08.879967 29398 raft_consensus.cc:348] T 3bceca7cc97447b8bbf4b7d07eb18598 P f7cfb1ad38554e1b927a9dad4015e35d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:08.880270 29398 raft_consensus.cc:374] T 3bceca7cc97447b8bbf4b7d07eb18598 P f7cfb1ad38554e1b927a9dad4015e35d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:08.880431 29398 raft_consensus.cc:729] T 3bceca7cc97447b8bbf4b7d07eb18598 P f7cfb1ad38554e1b927a9dad4015e35d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7cfb1ad38554e1b927a9dad4015e35d, State: Initialized, Role: FOLLOWER
I20260807 18:15:08.881083 29394 raft_consensus.cc:348] T 3bceca7cc97447b8bbf4b7d07eb18598 P 62030defec5b46a58dd672eeb50f5049 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:08.881443 29394 raft_consensus.cc:374] T 3bceca7cc97447b8bbf4b7d07eb18598 P 62030defec5b46a58dd672eeb50f5049 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:08.881629 29394 raft_consensus.cc:729] T 3bceca7cc97447b8bbf4b7d07eb18598 P 62030defec5b46a58dd672eeb50f5049 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62030defec5b46a58dd672eeb50f5049, State: Initialized, Role: FOLLOWER
I20260807 18:15:08.882090 29394 consensus_queue.cc:260] T 3bceca7cc97447b8bbf4b7d07eb18598 P 62030defec5b46a58dd672eeb50f5049 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:08.880713 29398 consensus_queue.cc:260] T 3bceca7cc97447b8bbf4b7d07eb18598 P f7cfb1ad38554e1b927a9dad4015e35d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:08.883972 29398 ts_tablet_manager.cc:1428] T 3bceca7cc97447b8bbf4b7d07eb18598 P f7cfb1ad38554e1b927a9dad4015e35d: Time spent starting tablet: real 0.005s user 0.001s sys 0.002s
I20260807 18:15:08.888047 29394 ts_tablet_manager.cc:1428] T 3bceca7cc97447b8bbf4b7d07eb18598 P 62030defec5b46a58dd672eeb50f5049: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260807 18:15:08.975274 29284 raft_consensus.cc:482] T 3bceca7cc97447b8bbf4b7d07eb18598 P f7cfb1ad38554e1b927a9dad4015e35d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:15:08.975648 29284 raft_consensus.cc:504] T 3bceca7cc97447b8bbf4b7d07eb18598 P f7cfb1ad38554e1b927a9dad4015e35d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:08.977939 29284 leader_election.cc:288] T 3bceca7cc97447b8bbf4b7d07eb18598 P f7cfb1ad38554e1b927a9dad4015e35d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bbc6335f70a54d61912f4ed30e2c6f79 (127.17.110.2:37795), 62030defec5b46a58dd672eeb50f5049 (127.17.110.3:36959)
I20260807 18:15:08.997589 29274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:08.978530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:58579 (local address 127.17.110.2:37795)
0807 18:15:08.978947 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:08.978956 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:08.989234 (+ 10278us) server_negotiation.cc:407] Connection header received
0807 18:15:08.989843 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:08.989854 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:08.989942 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:08.990013 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:15:08.990905 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.992158 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.994720 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.996181 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.996360 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:08.996442 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:08.997070 (+ 628us) server_negotiation.cc:299] Negotiation successful
0807 18:15:08.997276 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230}
I20260807 18:15:08.998653 29402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:08.978134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:37795 (local address 127.17.110.1:58579)
0807 18:15:08.989047 (+ 10913us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:08.989078 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:08.989627 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:08.990253 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:08.990258 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:08.990298 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:15:08.990663 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:08.990678 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.992369 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.992373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:08.994466 (+ 2093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:08.994487 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.996639 (+ 2152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.996646 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:08.996849 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:08.996886 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 18:15:08.998153 (+ 1267us) client_negotiation.cc:240] Negotiation successful
0807 18:15:08.998337 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10732,"thread_start_us":11118,"threads_started":1}
I20260807 18:15:08.999660 28976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bceca7cc97447b8bbf4b7d07eb18598" candidate_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" is_pre_election: true
I20260807 18:15:08.999969 28976 raft_consensus.cc:2454] T 3bceca7cc97447b8bbf4b7d07eb18598 P bbc6335f70a54d61912f4ed30e2c6f79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7cfb1ad38554e1b927a9dad4015e35d in term 0.
I20260807 18:15:09.000770 29348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:08.979351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:43951 (local address 127.17.110.3:36959)
0807 18:15:08.979580 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:08.979587 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:08.990530 (+ 10943us) server_negotiation.cc:407] Connection header received
0807 18:15:08.991092 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:08.991099 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:08.991216 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:08.991308 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:15:08.992246 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.993648 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.996899 (+ 3251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.998572 (+ 1673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.998816 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:08.998914 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:09.000171 (+ 1257us) server_negotiation.cc:299] Negotiation successful
0807 18:15:09.000366 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:15:09.000741 28699 leader_election.cc:302] T 3bceca7cc97447b8bbf4b7d07eb18598 P f7cfb1ad38554e1b927a9dad4015e35d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbc6335f70a54d61912f4ed30e2c6f79, f7cfb1ad38554e1b927a9dad4015e35d; no voters:
I20260807 18:15:09.001262 29284 raft_consensus.cc:2781] T 3bceca7cc97447b8bbf4b7d07eb18598 P f7cfb1ad38554e1b927a9dad4015e35d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:15:09.001386 29284 raft_consensus.cc:482] T 3bceca7cc97447b8bbf4b7d07eb18598 P f7cfb1ad38554e1b927a9dad4015e35d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:15:09.001478 29284 raft_consensus.cc:3037] T 3bceca7cc97447b8bbf4b7d07eb18598 P f7cfb1ad38554e1b927a9dad4015e35d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:09.003445 29403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:08.978487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:36959 (local address 127.17.110.1:43951)
0807 18:15:08.990352 (+ 11865us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:08.990379 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:08.990879 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:08.991547 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:08.991552 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:08.991584 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:15:08.991998 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:08.992031 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.994320 (+ 2289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.994329 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:08.996641 (+ 2312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:08.996662 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:08.999672 (+ 3010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:08.999680 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:08.999921 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:08.999964 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 18:15:09.002800 (+ 2836us) client_negotiation.cc:240] Negotiation successful
0807 18:15:09.003006 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11685,"thread_start_us":12056,"threads_started":1}
I20260807 18:15:09.004567 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bceca7cc97447b8bbf4b7d07eb18598" candidate_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62030defec5b46a58dd672eeb50f5049" is_pre_election: true
I20260807 18:15:09.004902 29175 raft_consensus.cc:2454] T 3bceca7cc97447b8bbf4b7d07eb18598 P 62030defec5b46a58dd672eeb50f5049 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7cfb1ad38554e1b927a9dad4015e35d in term 0.
I20260807 18:15:09.004998 29284 raft_consensus.cc:504] T 3bceca7cc97447b8bbf4b7d07eb18598 P f7cfb1ad38554e1b927a9dad4015e35d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:09.005829 29284 leader_election.cc:288] T 3bceca7cc97447b8bbf4b7d07eb18598 P f7cfb1ad38554e1b927a9dad4015e35d [CANDIDATE]: Term 1 election: Requested vote from peers bbc6335f70a54d61912f4ed30e2c6f79 (127.17.110.2:37795), 62030defec5b46a58dd672eeb50f5049 (127.17.110.3:36959)
I20260807 18:15:09.006506 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bceca7cc97447b8bbf4b7d07eb18598" candidate_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62030defec5b46a58dd672eeb50f5049"
I20260807 18:15:09.006790 29175 raft_consensus.cc:3037] T 3bceca7cc97447b8bbf4b7d07eb18598 P 62030defec5b46a58dd672eeb50f5049 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:09.007236 28976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bceca7cc97447b8bbf4b7d07eb18598" candidate_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbc6335f70a54d61912f4ed30e2c6f79"
I20260807 18:15:09.007498 28976 raft_consensus.cc:3037] T 3bceca7cc97447b8bbf4b7d07eb18598 P bbc6335f70a54d61912f4ed30e2c6f79 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:09.010741 29175 raft_consensus.cc:2454] T 3bceca7cc97447b8bbf4b7d07eb18598 P 62030defec5b46a58dd672eeb50f5049 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7cfb1ad38554e1b927a9dad4015e35d in term 1.
I20260807 18:15:09.011206 28976 raft_consensus.cc:2454] T 3bceca7cc97447b8bbf4b7d07eb18598 P bbc6335f70a54d61912f4ed30e2c6f79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7cfb1ad38554e1b927a9dad4015e35d in term 1.
I20260807 18:15:09.011400 28701 leader_election.cc:302] T 3bceca7cc97447b8bbf4b7d07eb18598 P f7cfb1ad38554e1b927a9dad4015e35d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62030defec5b46a58dd672eeb50f5049, f7cfb1ad38554e1b927a9dad4015e35d; no voters:
I20260807 18:15:09.011912 29284 raft_consensus.cc:2781] T 3bceca7cc97447b8bbf4b7d07eb18598 P f7cfb1ad38554e1b927a9dad4015e35d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:15:09.018152 29284 raft_consensus.cc:686] T 3bceca7cc97447b8bbf4b7d07eb18598 P f7cfb1ad38554e1b927a9dad4015e35d [term 1 LEADER]: Becoming Leader. State: Replica: f7cfb1ad38554e1b927a9dad4015e35d, State: Running, Role: LEADER
I20260807 18:15:09.018795 29284 consensus_queue.cc:237] T 3bceca7cc97447b8bbf4b7d07eb18598 P f7cfb1ad38554e1b927a9dad4015e35d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:09.029348 28570 catalog_manager.cc:5495] T 3bceca7cc97447b8bbf4b7d07eb18598 P f7cfb1ad38554e1b927a9dad4015e35d reported cstate change: term changed from 0 to 1, leader changed from <none> to f7cfb1ad38554e1b927a9dad4015e35d (127.17.110.1). New cstate: current_term: 1 leader_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } health_report { overall_health: UNKNOWN } } }
I20260807 18:15:09.063357 29407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:09.054195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56402 (local address 127.17.110.1:37549)
0807 18:15:09.055453 (+ 1258us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:09.055462 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:09.055489 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:15:09.055626 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:09.055632 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:09.055761 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:09.055968 (+ 207us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:15:09.056565 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:09.057827 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:09.060083 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:09.060757 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:09.060914 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:09.061596 (+ 682us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:15:09.062448 (+ 852us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:15:09.062615 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:09.062862 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 18:15:09.062962 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1099,"thread_start_us":921,"threads_started":1}
I20260807 18:15:09.068059 29175 raft_consensus.cc:1261] T 3bceca7cc97447b8bbf4b7d07eb18598 P 62030defec5b46a58dd672eeb50f5049 [term 1 FOLLOWER]: Refusing update from remote peer f7cfb1ad38554e1b927a9dad4015e35d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:15:09.068325 28976 raft_consensus.cc:1261] T 3bceca7cc97447b8bbf4b7d07eb18598 P bbc6335f70a54d61912f4ed30e2c6f79 [term 1 FOLLOWER]: Refusing update from remote peer f7cfb1ad38554e1b927a9dad4015e35d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:15:09.069029 29404 consensus_queue.cc:1035] T 3bceca7cc97447b8bbf4b7d07eb18598 P f7cfb1ad38554e1b927a9dad4015e35d [LEADER]: Connected to new peer: Peer: permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:15:09.069576 29284 consensus_queue.cc:1035] T 3bceca7cc97447b8bbf4b7d07eb18598 P f7cfb1ad38554e1b927a9dad4015e35d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:15:09.100394 29301 mvcc.cc:205] Tried to move back new op lower bound from 7315974181134192640 to 7315974180950102016. Current Snapshot: MvccSnapshot[applied={T|T < 7315974181134192640 or (T in {7315974181134192640})}]
I20260807 18:15:09.119021 28569 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:15:09.119396 28569 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:15:09.123555 28569 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f89b8e74b0ae4f2daab1653a7fbb4e00: Sending DeleteTablet for 3 replicas of tablet 3bceca7cc97447b8bbf4b7d07eb18598
I20260807 18:15:09.124889 29146 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bceca7cc97447b8bbf4b7d07eb18598 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:15:09 UTC) from {username='oryx', principal='oryx/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:48250
I20260807 18:15:09.125180 28945 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bceca7cc97447b8bbf4b7d07eb18598 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:15:09 UTC) from {username='oryx', principal='oryx/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:40028
I20260807 18:15:09.125532 29373 tablet_replica.cc:330] T 3bceca7cc97447b8bbf4b7d07eb18598 P bbc6335f70a54d61912f4ed30e2c6f79: stopping tablet replica
I20260807 18:15:09.125689 29372 tablet_replica.cc:330] T 3bceca7cc97447b8bbf4b7d07eb18598 P 62030defec5b46a58dd672eeb50f5049: stopping tablet replica
I20260807 18:15:09.125864 29373 raft_consensus.cc:2229] T 3bceca7cc97447b8bbf4b7d07eb18598 P bbc6335f70a54d61912f4ed30e2c6f79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:15:09.125844 28758 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bceca7cc97447b8bbf4b7d07eb18598 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:15:09 UTC) from {username='oryx', principal='oryx/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:56400
I20260807 18:15:09.125957 29372 raft_consensus.cc:2229] T 3bceca7cc97447b8bbf4b7d07eb18598 P 62030defec5b46a58dd672eeb50f5049 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:15:09.126142 29373 raft_consensus.cc:2258] T 3bceca7cc97447b8bbf4b7d07eb18598 P bbc6335f70a54d61912f4ed30e2c6f79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:15:09.126164 29370 tablet_replica.cc:330] T 3bceca7cc97447b8bbf4b7d07eb18598 P f7cfb1ad38554e1b927a9dad4015e35d: stopping tablet replica
I20260807 18:15:09.126245 29372 raft_consensus.cc:2258] T 3bceca7cc97447b8bbf4b7d07eb18598 P 62030defec5b46a58dd672eeb50f5049 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:15:09.126379 29370 raft_consensus.cc:2229] T 3bceca7cc97447b8bbf4b7d07eb18598 P f7cfb1ad38554e1b927a9dad4015e35d [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:15:09.127000 29370 raft_consensus.cc:2258] T 3bceca7cc97447b8bbf4b7d07eb18598 P f7cfb1ad38554e1b927a9dad4015e35d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:15:09.127450 29373 ts_tablet_manager.cc:1905] T 3bceca7cc97447b8bbf4b7d07eb18598 P bbc6335f70a54d61912f4ed30e2c6f79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:15:09.127601 28569 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50802:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 18:15:09.128034 29372 ts_tablet_manager.cc:1905] T 3bceca7cc97447b8bbf4b7d07eb18598 P 62030defec5b46a58dd672eeb50f5049: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 18:15:09.128448 28569 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:15:09.130924 29370 ts_tablet_manager.cc:1905] T 3bceca7cc97447b8bbf4b7d07eb18598 P f7cfb1ad38554e1b927a9dad4015e35d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:15:09.133702 29373 ts_tablet_manager.cc:1918] T 3bceca7cc97447b8bbf4b7d07eb18598 P bbc6335f70a54d61912f4ed30e2c6f79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:15:09.133978 29373 log.cc:1196] T 3bceca7cc97447b8bbf4b7d07eb18598 P bbc6335f70a54d61912f4ed30e2c6f79: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-1/wal/wals/3bceca7cc97447b8bbf4b7d07eb18598
I20260807 18:15:09.134424 29373 ts_tablet_manager.cc:1939] T 3bceca7cc97447b8bbf4b7d07eb18598 P bbc6335f70a54d61912f4ed30e2c6f79: Deleting consensus metadata
I20260807 18:15:09.136023 28551 catalog_manager.cc:4841] TS bbc6335f70a54d61912f4ed30e2c6f79 (127.17.110.2:37795): tablet 3bceca7cc97447b8bbf4b7d07eb18598 (table test-table [id=5423c742e4884a46a0859cf0e6e21297]) successfully deleted
I20260807 18:15:09.136346 29372 ts_tablet_manager.cc:1918] T 3bceca7cc97447b8bbf4b7d07eb18598 P 62030defec5b46a58dd672eeb50f5049: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:15:09.136547 29372 log.cc:1196] T 3bceca7cc97447b8bbf4b7d07eb18598 P 62030defec5b46a58dd672eeb50f5049: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-2/wal/wals/3bceca7cc97447b8bbf4b7d07eb18598
I20260807 18:15:09.137120 29372 ts_tablet_manager.cc:1939] T 3bceca7cc97447b8bbf4b7d07eb18598 P 62030defec5b46a58dd672eeb50f5049: Deleting consensus metadata
I20260807 18:15:09.138782 28553 catalog_manager.cc:4841] TS 62030defec5b46a58dd672eeb50f5049 (127.17.110.3:36959): tablet 3bceca7cc97447b8bbf4b7d07eb18598 (table test-table [id=5423c742e4884a46a0859cf0e6e21297]) successfully deleted
I20260807 18:15:09.144531 28758 tablet_service.cc:1467] Processing CreateTablet for tablet 63138d3b6f7445db87cb70bd981b841e (DEFAULT_TABLE table=test-table [id=a467e58a558a41528b8189e18dd41020]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:15:09.145107 28758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63138d3b6f7445db87cb70bd981b841e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:09.149266 28945 tablet_service.cc:1467] Processing CreateTablet for tablet 63138d3b6f7445db87cb70bd981b841e (DEFAULT_TABLE table=test-table [id=a467e58a558a41528b8189e18dd41020]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:15:09.149845 28945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63138d3b6f7445db87cb70bd981b841e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:09.156165 29146 tablet_service.cc:1467] Processing CreateTablet for tablet 63138d3b6f7445db87cb70bd981b841e (DEFAULT_TABLE table=test-table [id=a467e58a558a41528b8189e18dd41020]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:15:09.156873 29146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63138d3b6f7445db87cb70bd981b841e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:09.161478 29393 tablet_bootstrap.cc:492] T 63138d3b6f7445db87cb70bd981b841e P bbc6335f70a54d61912f4ed30e2c6f79: Bootstrap starting.
I20260807 18:15:09.163765 29393 tablet_bootstrap.cc:654] T 63138d3b6f7445db87cb70bd981b841e P bbc6335f70a54d61912f4ed30e2c6f79: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:09.164702 29394 tablet_bootstrap.cc:492] T 63138d3b6f7445db87cb70bd981b841e P 62030defec5b46a58dd672eeb50f5049: Bootstrap starting.
I20260807 18:15:09.165163 29370 ts_tablet_manager.cc:1918] T 3bceca7cc97447b8bbf4b7d07eb18598 P f7cfb1ad38554e1b927a9dad4015e35d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:15:09.166832 29394 tablet_bootstrap.cc:654] T 63138d3b6f7445db87cb70bd981b841e P 62030defec5b46a58dd672eeb50f5049: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:09.167116 29370 log.cc:1196] T 3bceca7cc97447b8bbf4b7d07eb18598 P f7cfb1ad38554e1b927a9dad4015e35d: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-0/wal/wals/3bceca7cc97447b8bbf4b7d07eb18598
I20260807 18:15:09.167742 29370 ts_tablet_manager.cc:1939] T 3bceca7cc97447b8bbf4b7d07eb18598 P f7cfb1ad38554e1b927a9dad4015e35d: Deleting consensus metadata
I20260807 18:15:09.169422 29398 tablet_bootstrap.cc:492] T 63138d3b6f7445db87cb70bd981b841e P f7cfb1ad38554e1b927a9dad4015e35d: Bootstrap starting.
I20260807 18:15:09.169407 28554 catalog_manager.cc:4841] TS f7cfb1ad38554e1b927a9dad4015e35d (127.17.110.1:37549): tablet 3bceca7cc97447b8bbf4b7d07eb18598 (table test-table [id=5423c742e4884a46a0859cf0e6e21297]) successfully deleted
I20260807 18:15:09.171434 29398 tablet_bootstrap.cc:654] T 63138d3b6f7445db87cb70bd981b841e P f7cfb1ad38554e1b927a9dad4015e35d: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:09.173661 29393 tablet_bootstrap.cc:492] T 63138d3b6f7445db87cb70bd981b841e P bbc6335f70a54d61912f4ed30e2c6f79: No bootstrap required, opened a new log
I20260807 18:15:09.173954 29393 ts_tablet_manager.cc:1397] T 63138d3b6f7445db87cb70bd981b841e P bbc6335f70a54d61912f4ed30e2c6f79: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.004s
I20260807 18:15:09.175001 29393 raft_consensus.cc:348] T 63138d3b6f7445db87cb70bd981b841e P bbc6335f70a54d61912f4ed30e2c6f79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:09.175480 29393 raft_consensus.cc:374] T 63138d3b6f7445db87cb70bd981b841e P bbc6335f70a54d61912f4ed30e2c6f79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:09.175638 29393 raft_consensus.cc:729] T 63138d3b6f7445db87cb70bd981b841e P bbc6335f70a54d61912f4ed30e2c6f79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbc6335f70a54d61912f4ed30e2c6f79, State: Initialized, Role: FOLLOWER
I20260807 18:15:09.176903 29393 consensus_queue.cc:260] T 63138d3b6f7445db87cb70bd981b841e P bbc6335f70a54d61912f4ed30e2c6f79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:09.179147 29393 ts_tablet_manager.cc:1428] T 63138d3b6f7445db87cb70bd981b841e P bbc6335f70a54d61912f4ed30e2c6f79: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260807 18:15:09.180222 29398 tablet_bootstrap.cc:492] T 63138d3b6f7445db87cb70bd981b841e P f7cfb1ad38554e1b927a9dad4015e35d: No bootstrap required, opened a new log
I20260807 18:15:09.180439 29398 ts_tablet_manager.cc:1397] T 63138d3b6f7445db87cb70bd981b841e P f7cfb1ad38554e1b927a9dad4015e35d: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.001s
I20260807 18:15:09.181316 29398 raft_consensus.cc:348] T 63138d3b6f7445db87cb70bd981b841e P f7cfb1ad38554e1b927a9dad4015e35d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:09.181602 29398 raft_consensus.cc:374] T 63138d3b6f7445db87cb70bd981b841e P f7cfb1ad38554e1b927a9dad4015e35d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:09.181697 29398 raft_consensus.cc:729] T 63138d3b6f7445db87cb70bd981b841e P f7cfb1ad38554e1b927a9dad4015e35d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7cfb1ad38554e1b927a9dad4015e35d, State: Initialized, Role: FOLLOWER
I20260807 18:15:09.181840 29394 tablet_bootstrap.cc:492] T 63138d3b6f7445db87cb70bd981b841e P 62030defec5b46a58dd672eeb50f5049: No bootstrap required, opened a new log
I20260807 18:15:09.182060 29394 ts_tablet_manager.cc:1397] T 63138d3b6f7445db87cb70bd981b841e P 62030defec5b46a58dd672eeb50f5049: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.006s
I20260807 18:15:09.181988 29398 consensus_queue.cc:260] T 63138d3b6f7445db87cb70bd981b841e P f7cfb1ad38554e1b927a9dad4015e35d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:09.182567 29398 ts_tablet_manager.cc:1428] T 63138d3b6f7445db87cb70bd981b841e P f7cfb1ad38554e1b927a9dad4015e35d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260807 18:15:09.183434 29394 raft_consensus.cc:348] T 63138d3b6f7445db87cb70bd981b841e P 62030defec5b46a58dd672eeb50f5049 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:09.183797 29394 raft_consensus.cc:374] T 63138d3b6f7445db87cb70bd981b841e P 62030defec5b46a58dd672eeb50f5049 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:09.183907 29394 raft_consensus.cc:729] T 63138d3b6f7445db87cb70bd981b841e P 62030defec5b46a58dd672eeb50f5049 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62030defec5b46a58dd672eeb50f5049, State: Initialized, Role: FOLLOWER
I20260807 18:15:09.184324 29394 consensus_queue.cc:260] T 63138d3b6f7445db87cb70bd981b841e P 62030defec5b46a58dd672eeb50f5049 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:09.185968 29394 ts_tablet_manager.cc:1428] T 63138d3b6f7445db87cb70bd981b841e P 62030defec5b46a58dd672eeb50f5049: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260807 18:15:09.328948 29284 raft_consensus.cc:482] T 63138d3b6f7445db87cb70bd981b841e P f7cfb1ad38554e1b927a9dad4015e35d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:15:09.329236 29284 raft_consensus.cc:504] T 63138d3b6f7445db87cb70bd981b841e P f7cfb1ad38554e1b927a9dad4015e35d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:09.330197 29284 leader_election.cc:288] T 63138d3b6f7445db87cb70bd981b841e P f7cfb1ad38554e1b927a9dad4015e35d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bbc6335f70a54d61912f4ed30e2c6f79 (127.17.110.2:37795), 62030defec5b46a58dd672eeb50f5049 (127.17.110.3:36959)
I20260807 18:15:09.330897 28976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63138d3b6f7445db87cb70bd981b841e" candidate_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" is_pre_election: true
I20260807 18:15:09.331205 28976 raft_consensus.cc:2454] T 63138d3b6f7445db87cb70bd981b841e P bbc6335f70a54d61912f4ed30e2c6f79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7cfb1ad38554e1b927a9dad4015e35d in term 0.
I20260807 18:15:09.331125 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63138d3b6f7445db87cb70bd981b841e" candidate_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62030defec5b46a58dd672eeb50f5049" is_pre_election: true
I20260807 18:15:09.331410 29175 raft_consensus.cc:2454] T 63138d3b6f7445db87cb70bd981b841e P 62030defec5b46a58dd672eeb50f5049 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7cfb1ad38554e1b927a9dad4015e35d in term 0.
I20260807 18:15:09.331954 28701 leader_election.cc:302] T 63138d3b6f7445db87cb70bd981b841e P f7cfb1ad38554e1b927a9dad4015e35d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 62030defec5b46a58dd672eeb50f5049, bbc6335f70a54d61912f4ed30e2c6f79, f7cfb1ad38554e1b927a9dad4015e35d; no voters:
I20260807 18:15:09.332436 29284 raft_consensus.cc:2781] T 63138d3b6f7445db87cb70bd981b841e P f7cfb1ad38554e1b927a9dad4015e35d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:15:09.332567 29284 raft_consensus.cc:482] T 63138d3b6f7445db87cb70bd981b841e P f7cfb1ad38554e1b927a9dad4015e35d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:15:09.332664 29284 raft_consensus.cc:3037] T 63138d3b6f7445db87cb70bd981b841e P f7cfb1ad38554e1b927a9dad4015e35d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:09.336459 29284 raft_consensus.cc:504] T 63138d3b6f7445db87cb70bd981b841e P f7cfb1ad38554e1b927a9dad4015e35d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:09.337808 29175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63138d3b6f7445db87cb70bd981b841e" candidate_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62030defec5b46a58dd672eeb50f5049"
I20260807 18:15:09.337831 28976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63138d3b6f7445db87cb70bd981b841e" candidate_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbc6335f70a54d61912f4ed30e2c6f79"
I20260807 18:15:09.338048 29175 raft_consensus.cc:3037] T 63138d3b6f7445db87cb70bd981b841e P 62030defec5b46a58dd672eeb50f5049 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:09.338090 28976 raft_consensus.cc:3037] T 63138d3b6f7445db87cb70bd981b841e P bbc6335f70a54d61912f4ed30e2c6f79 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:09.341361 29175 raft_consensus.cc:2454] T 63138d3b6f7445db87cb70bd981b841e P 62030defec5b46a58dd672eeb50f5049 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7cfb1ad38554e1b927a9dad4015e35d in term 1.
I20260807 18:15:09.341928 28701 leader_election.cc:302] T 63138d3b6f7445db87cb70bd981b841e P f7cfb1ad38554e1b927a9dad4015e35d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62030defec5b46a58dd672eeb50f5049, f7cfb1ad38554e1b927a9dad4015e35d; no voters:
I20260807 18:15:09.342303 29414 raft_consensus.cc:2781] T 63138d3b6f7445db87cb70bd981b841e P f7cfb1ad38554e1b927a9dad4015e35d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:15:09.342505 29414 raft_consensus.cc:686] T 63138d3b6f7445db87cb70bd981b841e P f7cfb1ad38554e1b927a9dad4015e35d [term 1 LEADER]: Becoming Leader. State: Replica: f7cfb1ad38554e1b927a9dad4015e35d, State: Running, Role: LEADER
I20260807 18:15:09.342890 29414 consensus_queue.cc:237] T 63138d3b6f7445db87cb70bd981b841e P f7cfb1ad38554e1b927a9dad4015e35d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } }
I20260807 18:15:09.343101 28976 raft_consensus.cc:2454] T 63138d3b6f7445db87cb70bd981b841e P bbc6335f70a54d61912f4ed30e2c6f79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7cfb1ad38554e1b927a9dad4015e35d in term 1.
I20260807 18:15:09.345635 29284 leader_election.cc:288] T 63138d3b6f7445db87cb70bd981b841e P f7cfb1ad38554e1b927a9dad4015e35d [CANDIDATE]: Term 1 election: Requested vote from peers bbc6335f70a54d61912f4ed30e2c6f79 (127.17.110.2:37795), 62030defec5b46a58dd672eeb50f5049 (127.17.110.3:36959)
I20260807 18:15:09.346612 28568 catalog_manager.cc:5495] T 63138d3b6f7445db87cb70bd981b841e P f7cfb1ad38554e1b927a9dad4015e35d reported cstate change: term changed from 0 to 1, leader changed from <none> to f7cfb1ad38554e1b927a9dad4015e35d (127.17.110.1). New cstate: current_term: 1 leader_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7cfb1ad38554e1b927a9dad4015e35d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 37549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 } health_report { overall_health: UNKNOWN } } }
I20260807 18:15:09.395110 29274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:09.389128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:45615 (local address 127.17.110.2:37795)
0807 18:15:09.389318 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:09.389323 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:09.389859 (+ 536us) server_negotiation.cc:407] Connection header received
0807 18:15:09.390097 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:09.390103 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:09.390175 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:09.390274 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:15:09.390886 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:09.391659 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:09.393183 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:09.393646 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:09.393792 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:09.394270 (+ 478us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:15:09.394387 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:15:09.394484 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:09.394643 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 18:15:09.394766 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:15:09.395292 29437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:09.388974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:37795 (local address 127.17.110.1:45615)
0807 18:15:09.389741 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:09.389757 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:09.389946 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:09.390294 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:09.390299 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:09.390311 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:15:09.390680 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:09.390690 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:09.391797 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:09.391803 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:09.393028 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:09.393042 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:09.393856 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:09.393864 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:09.394002 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:09.394068 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:15:09.394488 (+ 420us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:15:09.394505 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 18:15:09.394597 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 18:15:09.394777 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":454,"threads_started":1}
I20260807 18:15:09.405028 28976 raft_consensus.cc:1261] T 63138d3b6f7445db87cb70bd981b841e P bbc6335f70a54d61912f4ed30e2c6f79 [term 1 FOLLOWER]: Refusing update from remote peer f7cfb1ad38554e1b927a9dad4015e35d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:15:09.405093 29175 raft_consensus.cc:1261] T 63138d3b6f7445db87cb70bd981b841e P 62030defec5b46a58dd672eeb50f5049 [term 1 FOLLOWER]: Refusing update from remote peer f7cfb1ad38554e1b927a9dad4015e35d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:15:09.405748 29414 consensus_queue.cc:1035] T 63138d3b6f7445db87cb70bd981b841e P f7cfb1ad38554e1b927a9dad4015e35d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbc6335f70a54d61912f4ed30e2c6f79" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 37795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:15:09.406152 29284 consensus_queue.cc:1035] T 63138d3b6f7445db87cb70bd981b841e P f7cfb1ad38554e1b927a9dad4015e35d [LEADER]: Connected to new peer: Peer: permanent_uuid: "62030defec5b46a58dd672eeb50f5049" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:15:09.452836 29407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:09.440083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:59977 (local address 127.17.110.1:37549)
0807 18:15:09.440354 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:09.440364 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:09.446692 (+ 6328us) server_negotiation.cc:407] Connection header received
0807 18:15:09.446940 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:09.446947 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:09.447068 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:09.447188 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:15:09.447753 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:09.448789 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:09.450571 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:09.451352 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:09.451523 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:09.451931 (+ 408us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:15:09.452111 (+ 180us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:15:09.452219 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:09.452384 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 18:15:09.452474 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260807 18:15:09.452956 29444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:09.440098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:37549 (local address 127.17.110.2:59977)
0807 18:15:09.446542 (+ 6444us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:09.446565 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:09.446767 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:09.447246 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:09.447251 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:09.447265 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:15:09.447558 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:09.447574 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:09.449037 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:09.449044 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:09.450396 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:09.450413 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:09.451535 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:09.451540 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:09.451672 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:09.451738 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:15:09.452228 (+ 490us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:15:09.452242 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 18:15:09.452335 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 18:15:09.452490 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6267,"thread_start_us":6118,"threads_started":1}
I20260807 18:15:09.492244 28568 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:15:09.492491 28568 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:15:09.496380 28568 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f89b8e74b0ae4f2daab1653a7fbb4e00: Sending DeleteTablet for 3 replicas of tablet 63138d3b6f7445db87cb70bd981b841e
I20260807 18:15:09.497500 28945 tablet_service.cc:1514] Processing DeleteTablet for tablet 63138d3b6f7445db87cb70bd981b841e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:15:09 UTC) from {username='oryx', principal='oryx/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:40028
I20260807 18:15:09.497545 28758 tablet_service.cc:1514] Processing DeleteTablet for tablet 63138d3b6f7445db87cb70bd981b841e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:15:09 UTC) from {username='oryx', principal='oryx/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:56400
I20260807 18:15:09.498019 29370 tablet_replica.cc:330] T 63138d3b6f7445db87cb70bd981b841e P f7cfb1ad38554e1b927a9dad4015e35d: stopping tablet replica
I20260807 18:15:09.498157 29373 tablet_replica.cc:330] T 63138d3b6f7445db87cb70bd981b841e P bbc6335f70a54d61912f4ed30e2c6f79: stopping tablet replica
I20260807 18:15:09.498193 29146 tablet_service.cc:1514] Processing DeleteTablet for tablet 63138d3b6f7445db87cb70bd981b841e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:15:09 UTC) from {username='oryx', principal='oryx/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:48250
I20260807 18:15:09.498286 29370 raft_consensus.cc:2229] T 63138d3b6f7445db87cb70bd981b841e P f7cfb1ad38554e1b927a9dad4015e35d [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:15:09.498339 29373 raft_consensus.cc:2229] T 63138d3b6f7445db87cb70bd981b841e P bbc6335f70a54d61912f4ed30e2c6f79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:15:09.498513 29372 tablet_replica.cc:330] T 63138d3b6f7445db87cb70bd981b841e P 62030defec5b46a58dd672eeb50f5049: stopping tablet replica
I20260807 18:15:09.498569 29373 raft_consensus.cc:2258] T 63138d3b6f7445db87cb70bd981b841e P bbc6335f70a54d61912f4ed30e2c6f79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:15:09.498739 29370 raft_consensus.cc:2258] T 63138d3b6f7445db87cb70bd981b841e P f7cfb1ad38554e1b927a9dad4015e35d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:15:09.498744 29372 raft_consensus.cc:2229] T 63138d3b6f7445db87cb70bd981b841e P 62030defec5b46a58dd672eeb50f5049 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:15:09.498972 29372 raft_consensus.cc:2258] T 63138d3b6f7445db87cb70bd981b841e P 62030defec5b46a58dd672eeb50f5049 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:15:09.500691 29370 ts_tablet_manager.cc:1905] T 63138d3b6f7445db87cb70bd981b841e P f7cfb1ad38554e1b927a9dad4015e35d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:15:09.501150 29372 ts_tablet_manager.cc:1905] T 63138d3b6f7445db87cb70bd981b841e P 62030defec5b46a58dd672eeb50f5049: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:15:09.501230 29373 ts_tablet_manager.cc:1905] T 63138d3b6f7445db87cb70bd981b841e P bbc6335f70a54d61912f4ed30e2c6f79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:15:09.508874 29373 ts_tablet_manager.cc:1918] T 63138d3b6f7445db87cb70bd981b841e P bbc6335f70a54d61912f4ed30e2c6f79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:15:09.509083 29373 log.cc:1196] T 63138d3b6f7445db87cb70bd981b841e P bbc6335f70a54d61912f4ed30e2c6f79: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-1/wal/wals/63138d3b6f7445db87cb70bd981b841e
I20260807 18:15:09.509649 29373 ts_tablet_manager.cc:1939] T 63138d3b6f7445db87cb70bd981b841e P bbc6335f70a54d61912f4ed30e2c6f79: Deleting consensus metadata
I20260807 18:15:09.511112 29372 ts_tablet_manager.cc:1918] T 63138d3b6f7445db87cb70bd981b841e P 62030defec5b46a58dd672eeb50f5049: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:15:09.511104 28551 catalog_manager.cc:4841] TS bbc6335f70a54d61912f4ed30e2c6f79 (127.17.110.2:37795): tablet 63138d3b6f7445db87cb70bd981b841e (table test-table [id=a467e58a558a41528b8189e18dd41020]) successfully deleted
I20260807 18:15:09.511412 29372 log.cc:1196] T 63138d3b6f7445db87cb70bd981b841e P 62030defec5b46a58dd672eeb50f5049: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-2/wal/wals/63138d3b6f7445db87cb70bd981b841e
I20260807 18:15:09.511689 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 28677
I20260807 18:15:09.511932 29372 ts_tablet_manager.cc:1939] T 63138d3b6f7445db87cb70bd981b841e P 62030defec5b46a58dd672eeb50f5049: Deleting consensus metadata
I20260807 18:15:09.513513 29370 ts_tablet_manager.cc:1918] T 63138d3b6f7445db87cb70bd981b841e P f7cfb1ad38554e1b927a9dad4015e35d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:15:09.513707 29370 log.cc:1196] T 63138d3b6f7445db87cb70bd981b841e P f7cfb1ad38554e1b927a9dad4015e35d: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126429568977-17848-0/minicluster-data/ts-0/wal/wals/63138d3b6f7445db87cb70bd981b841e
I20260807 18:15:09.513985 28553 catalog_manager.cc:4841] TS 62030defec5b46a58dd672eeb50f5049 (127.17.110.3:36959): tablet 63138d3b6f7445db87cb70bd981b841e (table test-table [id=a467e58a558a41528b8189e18dd41020]) successfully deleted
I20260807 18:15:09.514606 29370 ts_tablet_manager.cc:1939] T 63138d3b6f7445db87cb70bd981b841e P f7cfb1ad38554e1b927a9dad4015e35d: Deleting consensus metadata
I20260807 18:15:09.515748 28554 catalog_manager.cc:4841] TS f7cfb1ad38554e1b927a9dad4015e35d (127.17.110.1:37549): tablet 63138d3b6f7445db87cb70bd981b841e (table test-table [id=a467e58a558a41528b8189e18dd41020]) successfully deleted
Aug 07 18:15:09 dist-test-slave-39l8 krb5kdc[28513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.110.1@KRBTEST.COM
I20260807 18:15:09.541103 29407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:09.513012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56410 (local address 127.17.110.1:37549)
0807 18:15:09.513276 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:09.513283 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:09.514247 (+ 964us) server_negotiation.cc:407] Connection header received
0807 18:15:09.514586 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:09.514598 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:09.514695 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:09.514861 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:09.516608 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:09.517349 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:09.519319 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:09.519805 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:09.519913 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:09.532699 (+ 12786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:09.532716 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:09.532732 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:09.532781 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:09.537436 (+ 4655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:09.538370 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:09.538380 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:09.538384 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:09.538552 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:09.539028 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:09.539034 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:09.539038 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:09.539506 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:09.540015 (+ 509us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:09.540346 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 18:15:09.540630 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260807 18:15:09.541991 28847 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:09.680806 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 28677
I20260807 18:15:09.714620 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 28862
Aug 07 18:15:09 dist-test-slave-39l8 krb5kdc[28513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.110.2@KRBTEST.COM
I20260807 18:15:09.732924 29274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:09.715588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40060 (local address 127.17.110.2:37795)
0807 18:15:09.715790 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:09.715804 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:09.715828 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:15:09.715907 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:09.715912 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:09.715974 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:09.716083 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:09.717360 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:09.718036 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:09.719602 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:09.720010 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:09.720197 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:09.725107 (+ 4910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:09.725129 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:09.725147 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:09.725219 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:09.729343 (+ 4124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:09.730248 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:09.730257 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:09.730260 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:09.730446 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:09.730982 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:09.730987 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:09.730989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:09.731422 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:09.731801 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:09.732248 (+ 447us) server_negotiation.cc:299] Negotiation successful
0807 18:15:09.732533 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:15:09.733896 29039 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:09.895958 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 28862
I20260807 18:15:09.932341 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 29056
Aug 07 18:15:09 dist-test-slave-39l8 krb5kdc[28513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.110.3@KRBTEST.COM
I20260807 18:15:09.951179 29451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:09.933323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48274 (local address 127.17.110.3:36959)
0807 18:15:09.933928 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:09.933933 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:09.933950 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:15:09.934007 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:09.934010 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:09.934085 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:09.934186 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:09.935904 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:09.936692 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:09.938140 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:09.938593 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:09.938887 (+ 294us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:09.944266 (+ 5379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:09.944287 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:09.944306 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:09.944355 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:09.947614 (+ 3259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:09.948573 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:09.948582 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:09.948584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:09.948810 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:09.949338 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:09.949342 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:09.949345 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:09.949858 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:09.950162 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:09.950579 (+ 417us) server_negotiation.cc:299] Negotiation successful
0807 18:15:09.950884 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":329,"threads_started":1}
I20260807 18:15:09.951970 29239 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:10.091549 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 29056
I20260807 18:15:10.119820 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 28529
I20260807 18:15:10.121271 28634 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:10.241089 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 28529
2026-08-07T18:15:10Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (8428 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:15:10 dist-test-slave-39l8 krb5kdc[29459](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:15:10 dist-test-slave-39l8 krb5kdc[29459](info): set up 2 sockets
Aug 07 18:15:10 dist-test-slave-39l8 krb5kdc[29459](info): commencing operation
krb5kdc: starting...
W20260807 18:15:11.579363 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.281s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:15:11 dist-test-slave-39l8 krb5kdc[29459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:15:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:15:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.110.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
I20260807 18:15:11.876864 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:43181
--webserver_interface=127.17.110.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:43181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:15:12.239347 29475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:12.239737 29475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:12.239908 29475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:12.247402 29475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:15:12.247573 29475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:12.247673 29475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:12.247774 29475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:15:12.247861 29475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:15:12.258106 29475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:43181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:43181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:15:12.259671 29475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:12.261083 29475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:12.275349 29482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:12.275339 29486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:12.275395 29484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:12.276331 29475 server_base.cc:1034] running on GCE node
I20260807 18:15:12.277350 29475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:12.279131 29475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:12.280385 29475 hybrid_clock.cc:648] HybridClock initialized: now 1786126512280331 us; error 36 us; skew 500 ppm
I20260807 18:15:12.280836 29475 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T18:15:12Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2111 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:15:12 dist-test-slave-39l8 krb5kdc[29494](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:15:12 dist-test-slave-39l8 krb5kdc[29494](info): set up 2 sockets
Aug 07 18:15:12 dist-test-slave-39l8 krb5kdc[29494](info): commencing operation
krb5kdc: starting...
W20260807 18:15:13.745208 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.335s user 0.002s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:15:13 dist-test-slave-39l8 krb5kdc[29494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:15:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:15:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.110.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
I20260807 18:15:14.059020 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:38611
--webserver_interface=127.17.110.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:35207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:38611
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:15:14.417203 29512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:14.417536 29512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:14.417630 29512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:14.425130 29512 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:15:14.425267 29512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:14.425315 29512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:14.425376 29512 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:15:14.425421 29512 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:15:14.435789 29512 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:35207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:38611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:38611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:15:14.437177 29512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:14.438503 29512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:14.451012 29518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:14.451154 29519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:14.451782 29523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:14.452173 29512 server_base.cc:1034] running on GCE node
I20260807 18:15:14.453097 29512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:14.454692 29512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:14.456004 29512 hybrid_clock.cc:648] HybridClock initialized: now 1786126514455932 us; error 64 us; skew 500 ppm
I20260807 18:15:14.456406 29512 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:15:14 dist-test-slave-39l8 krb5kdc[29494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126514, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:14.460000 29512 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:15:14.461651 29512 webserver.cc:460] Webserver started at http://127.17.110.62:46173/ using document root <none> and password file <none>
I20260807 18:15:14.462455 29512 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:14.462611 29512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:14.463009 29512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:15:14.466282 29512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "dce360763d52470b9281fe028082ac03"
format_stamp: "Formatted at 2026-08-07 18:15:14 on dist-test-slave-39l8"
server_key: "85f3513ce8951b9753252e23f14991c5"
server_key_iv: "822b50fd9b983b35ffdcdeee2a28d387"
server_key_version: "encryptionkey@0"
I20260807 18:15:14.467182 29512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "dce360763d52470b9281fe028082ac03"
format_stamp: "Formatted at 2026-08-07 18:15:14 on dist-test-slave-39l8"
server_key: "85f3513ce8951b9753252e23f14991c5"
server_key_iv: "822b50fd9b983b35ffdcdeee2a28d387"
server_key_version: "encryptionkey@0"
I20260807 18:15:14.473220 29512 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 18:15:14.477077 29533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:14.478649 29512 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:15:14.478901 29512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "dce360763d52470b9281fe028082ac03"
format_stamp: "Formatted at 2026-08-07 18:15:14 on dist-test-slave-39l8"
server_key: "85f3513ce8951b9753252e23f14991c5"
server_key_iv: "822b50fd9b983b35ffdcdeee2a28d387"
server_key_version: "encryptionkey@0"
I20260807 18:15:14.479103 29512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:14.540297 29512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:14.546027 29512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:14.546340 29512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:14.568307 29512 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:38611
I20260807 18:15:14.568326 29610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:38611 every 8 connection(s)
I20260807 18:15:14.570070 29512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:15:14.574263 29611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:14.580024 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 29512
I20260807 18:15:14.580334 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:15:14.581032 17848 external_mini_cluster.cc:1428] Setting key afd97b16c2bf31bd790f0409db63bbef
I20260807 18:15:14.587783 29611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03: Bootstrap starting.
I20260807 18:15:14.591866 29611 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:14.593032 29611 log.cc:824] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03: Log is configured to *not* fsync() on all Append() calls
I20260807 18:15:14.596544 29611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03: No bootstrap required, opened a new log
Aug 07 18:15:14 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:15:14.607658 29611 raft_consensus.cc:348] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dce360763d52470b9281fe028082ac03" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38611 } }
I20260807 18:15:14.608199 29611 raft_consensus.cc:374] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:14.608314 29611 raft_consensus.cc:729] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dce360763d52470b9281fe028082ac03, State: Initialized, Role: FOLLOWER
I20260807 18:15:14.609066 29611 consensus_queue.cc:260] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dce360763d52470b9281fe028082ac03" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38611 } }
I20260807 18:15:14.609387 29611 raft_consensus.cc:388] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:15:14.609489 29617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:14.583442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50178 (local address 127.17.110.62:38611)
0807 18:15:14.584538 (+ 1096us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:14.584556 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:14.584613 (+ 57us) server_negotiation.cc:407] Connection header received
0807 18:15:14.587238 (+ 2625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:14.587277 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:14.587848 (+ 571us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:14.588342 (+ 494us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:14.589841 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:14.591910 (+ 2069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:14.593311 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:14.594279 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:14.594447 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:14.599311 (+ 4864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:14.599323 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:14.599347 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:14.599409 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:14.604002 (+ 4593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:14.604919 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:14.604935 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:14.604946 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:14.605219 (+ 273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:14.605728 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:14.605735 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:14.605738 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:14.606581 (+ 843us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:14.606918 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:14.607482 (+ 564us) server_negotiation.cc:299] Negotiation successful
0807 18:15:14.607984 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":808,"thread_start_us":516,"threads_started":1}
I20260807 18:15:14.609570 29611 raft_consensus.cc:482] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:15:14.609781 29611 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:14.612746 29611 raft_consensus.cc:504] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dce360763d52470b9281fe028082ac03" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38611 } }
I20260807 18:15:14.613358 29611 leader_election.cc:302] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dce360763d52470b9281fe028082ac03; no voters:
I20260807 18:15:14.613994 29611 leader_election.cc:288] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:15:14.614187 29619 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:15:14.614982 29619 raft_consensus.cc:686] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [term 1 LEADER]: Becoming Leader. State: Replica: dce360763d52470b9281fe028082ac03, State: Running, Role: LEADER
I20260807 18:15:14.615583 29619 consensus_queue.cc:237] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dce360763d52470b9281fe028082ac03" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38611 } }
I20260807 18:15:14.616482 29611 sys_catalog.cc:564] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:15:14.618788 29620 sys_catalog.cc:455] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dce360763d52470b9281fe028082ac03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dce360763d52470b9281fe028082ac03" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38611 } } }
I20260807 18:15:14.619571 29620 sys_catalog.cc:458] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [sys.catalog]: This master's current role is: LEADER
I20260807 18:15:14.618845 29622 sys_catalog.cc:455] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dce360763d52470b9281fe028082ac03. Latest consensus state: current_term: 1 leader_uuid: "dce360763d52470b9281fe028082ac03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dce360763d52470b9281fe028082ac03" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38611 } } }
I20260807 18:15:14.620289 29622 sys_catalog.cc:458] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [sys.catalog]: This master's current role is: LEADER
I20260807 18:15:14.624477 29629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:15:14.632740 29629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:15:14.642501 29629 catalog_manager.cc:1347] Generated new cluster ID: a0ecce4b3b394f5389bd177db17259af
I20260807 18:15:14.642643 29629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:15:14.653855 29629 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:15:14.656050 29629 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:15:14.669176 29629 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03: Generated new TSK 0
I20260807 18:15:14.670497 29629 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.110.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
I20260807 18:15:14.750998 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.1:0
--local_ip_for_outbound_sockets=127.17.110.1
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:38611
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:35207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:15:15.119421 29652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:15.119809 29652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:15.119979 29652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:15.127753 29652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:15.127923 29652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:15.128134 29652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.1
I20260807 18:15:15.139035 29652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:35207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:38611
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.1
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:15:15.140482 29652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:15.141878 29652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:15.156392 29662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:15.156364 29665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:15.156448 29660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:15.157171 29652 server_base.cc:1034] running on GCE node
I20260807 18:15:15.157909 29652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:15.159088 29652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:15.160346 29652 hybrid_clock.cc:648] HybridClock initialized: now 1786126515160295 us; error 48 us; skew 500 ppm
I20260807 18:15:15.160727 29652 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:15:15 dist-test-slave-39l8 krb5kdc[29494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126515, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:15.164460 29652 init.cc:377] Logged in from keytab as kudu/127.17.110.1@KRBTEST.COM (short username kudu)
I20260807 18:15:15.166162 29652 webserver.cc:460] Webserver started at http://127.17.110.1:42059/ using document root <none> and password file <none>
I20260807 18:15:15.166985 29652 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:15.167097 29652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:15.167382 29652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:15:15.170773 29652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-0/data/instance:
uuid: "c74dbc44bf1b484bb643820f64c42cad"
format_stamp: "Formatted at 2026-08-07 18:15:15 on dist-test-slave-39l8"
server_key: "6b4517793d3ae2a57e9e10616d47d443"
server_key_iv: "1c82d5bac67b92564ee0a7a61d55ce1a"
server_key_version: "encryptionkey@0"
I20260807 18:15:15.171645 29652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance:
uuid: "c74dbc44bf1b484bb643820f64c42cad"
format_stamp: "Formatted at 2026-08-07 18:15:15 on dist-test-slave-39l8"
server_key: "6b4517793d3ae2a57e9e10616d47d443"
server_key_iv: "1c82d5bac67b92564ee0a7a61d55ce1a"
server_key_version: "encryptionkey@0"
I20260807 18:15:15.177366 29652 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:15:15.181152 29674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:15.182934 29652 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:15:15.183142 29652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-0/wal
uuid: "c74dbc44bf1b484bb643820f64c42cad"
format_stamp: "Formatted at 2026-08-07 18:15:15 on dist-test-slave-39l8"
server_key: "6b4517793d3ae2a57e9e10616d47d443"
server_key_iv: "1c82d5bac67b92564ee0a7a61d55ce1a"
server_key_version: "encryptionkey@0"
I20260807 18:15:15.183386 29652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:15.206831 29652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:15.212193 29652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:15.212538 29652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:15.213642 29652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:15:15.215163 29652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:15:15.215246 29652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:15.215413 29652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:15:15.215484 29652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:15.252926 29652 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.1:38351
I20260807 18:15:15.253032 29848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.1:38351 every 8 connection(s)
I20260807 18:15:15.254329 29652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
I20260807 18:15:15.256966 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 29652
I20260807 18:15:15.257215 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance
I20260807 18:15:15.258124 17848 external_mini_cluster.cc:1428] Setting key 416f3d531710c88f54b43a4b476dfe69
Aug 07 18:15:15 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126515, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:15:15.284698 29851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:15.257744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:60981 (local address 127.17.110.62:38611)
0807 18:15:15.258340 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:15.258349 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:15.259532 (+ 1183us) server_negotiation.cc:407] Connection header received
0807 18:15:15.262830 (+ 3298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:15.262835 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:15.262933 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:15.263036 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:15.265832 (+ 2796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:15.266588 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:15.269808 (+ 3220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:15.270394 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:15.270552 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:15.275881 (+ 5329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:15.275891 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:15.275900 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:15.275955 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:15.280106 (+ 4151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:15.281170 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:15.281179 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:15.281186 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:15.281367 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:15.281878 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:15.281886 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:15.281891 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:15.282405 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:15.282576 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:15.283810 (+ 1234us) server_negotiation.cc:299] Negotiation successful
0807 18:15:15.284187 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":283,"threads_started":1}
I20260807 18:15:15.285642 29852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:15.258394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.1:60981)
0807 18:15:15.259333 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:15.259390 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:15.262233 (+ 2843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:15.263427 (+ 1194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:15.263445 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:15.264508 (+ 1063us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:15.265538 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:15.265574 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:15.266863 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:15.266872 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:15.269482 (+ 2610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:15.269510 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:15.270757 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:15.270772 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:15.271199 (+ 427us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:15.272212 (+ 1013us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:15.272256 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:15.275525 (+ 3269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:15.280380 (+ 4855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:15.280392 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:15.280410 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:15.281001 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:15.281510 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:15.281514 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:15.281516 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:15.281741 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:15.282576 (+ 835us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:15.282584 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:15.283187 (+ 603us) client_negotiation.cc:769] Sending connection context
0807 18:15:15.283640 (+ 453us) client_negotiation.cc:240] Negotiation successful
0807 18:15:15.284320 (+ 680us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":376,"threads_started":1}
I20260807 18:15:15.287601 29849 heartbeater.cc:347] Connected to a master server at 127.17.110.62:38611
I20260807 18:15:15.287986 29849 heartbeater.cc:464] Registering TS with master...
I20260807 18:15:15.288921 29849 heartbeater.cc:511] Master 127.17.110.62:38611 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.17.110.2@KRBTEST.COM; defaulting to no policy
I20260807 18:15:15.291543 29558 ts_manager.cc:194] Registered new tserver with Master: c74dbc44bf1b484bb643820f64c42cad (127.17.110.1:38351)
I20260807 18:15:15.293628 29558 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.1@KRBTEST.COM'} at 127.17.110.1:60981
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
I20260807 18:15:15.331521 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.2:0
--local_ip_for_outbound_sockets=127.17.110.2
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:38611
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:35207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:15:15.715585 29859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:15.715936 29859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:15.716107 29859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:15.724725 29859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:15.724855 29859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:15.724972 29859 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.2
I20260807 18:15:15.735304 29859 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:35207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:38611
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.2
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:15:15.736622 29859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:15.737947 29859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:15.751663 29866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:15.751694 29867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:15.751869 29873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:15.752638 29859 server_base.cc:1034] running on GCE node
I20260807 18:15:15.753336 29859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:15.754318 29859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:15.755518 29859 hybrid_clock.cc:648] HybridClock initialized: now 1786126515755460 us; error 58 us; skew 500 ppm
I20260807 18:15:15.755843 29859 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:15:15 dist-test-slave-39l8 krb5kdc[29494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126515, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:15.759135 29859 init.cc:377] Logged in from keytab as kudu/127.17.110.2@KRBTEST.COM (short username kudu)
I20260807 18:15:15.760767 29859 webserver.cc:460] Webserver started at http://127.17.110.2:40267/ using document root <none> and password file <none>
I20260807 18:15:15.761463 29859 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:15.761567 29859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:15.761847 29859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:15:15.764961 29859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-1/data/instance:
uuid: "9b23c0f33641401193a33a09fd13b218"
format_stamp: "Formatted at 2026-08-07 18:15:15 on dist-test-slave-39l8"
server_key: "6804726fa4c7a28351e7c4729133bb73"
server_key_iv: "c82886019cf74418dd5829b40da2d9db"
server_key_version: "encryptionkey@0"
I20260807 18:15:15.765753 29859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance:
uuid: "9b23c0f33641401193a33a09fd13b218"
format_stamp: "Formatted at 2026-08-07 18:15:15 on dist-test-slave-39l8"
server_key: "6804726fa4c7a28351e7c4729133bb73"
server_key_iv: "c82886019cf74418dd5829b40da2d9db"
server_key_version: "encryptionkey@0"
I20260807 18:15:15.771467 29859 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 18:15:15.775817 29882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:15.777697 29859 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:15:15.777951 29859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-1/wal
uuid: "9b23c0f33641401193a33a09fd13b218"
format_stamp: "Formatted at 2026-08-07 18:15:15 on dist-test-slave-39l8"
server_key: "6804726fa4c7a28351e7c4729133bb73"
server_key_iv: "c82886019cf74418dd5829b40da2d9db"
server_key_version: "encryptionkey@0"
I20260807 18:15:15.778239 29859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:15.799808 29859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:15.806226 29859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:15.806617 29859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:15.807886 29859 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:15:15.809814 29859 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:15:15.809938 29859 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:15.810114 29859 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:15:15.810196 29859 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:15.850826 29859 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.2:39667
I20260807 18:15:15.850919 30044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.2:39667 every 8 connection(s)
I20260807 18:15:15.852305 29859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
I20260807 18:15:15.855703 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 29859
I20260807 18:15:15.855934 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance
I20260807 18:15:15.856472 17848 external_mini_cluster.cc:1428] Setting key 422e58458eed88a97bcdee58bb199159
Aug 07 18:15:15 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126515, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:15:15.878619 30051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:15.855521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:51707 (local address 127.17.110.62:38611)
0807 18:15:15.856109 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:15.856117 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:15.857409 (+ 1292us) server_negotiation.cc:407] Connection header received
0807 18:15:15.859630 (+ 2221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:15.859640 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:15.859744 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:15.859870 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:15.861694 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:15.863000 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:15.864798 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:15.865635 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:15.865794 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:15.869854 (+ 4060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:15.869863 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:15.869873 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:15.869929 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:15.874207 (+ 4278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:15.875481 (+ 1274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:15.875490 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:15.875497 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:15.875684 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:15.876232 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:15.876242 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:15.876248 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:15.876713 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:15.876892 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:15.877725 (+ 833us) server_negotiation.cc:299] Negotiation successful
0807 18:15:15.878063 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":333,"threads_started":1}
I20260807 18:15:15.879380 30052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:15.856186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.2:51707)
0807 18:15:15.857160 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:15.857237 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:15.859308 (+ 2071us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:15.860129 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:15.860139 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:15.860813 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:15.861483 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:15.861503 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:15.863189 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:15.863194 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:15.864630 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:15.864644 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:15.865900 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:15.865906 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:15.866128 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:15.866988 (+ 860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:15.867022 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:15.869521 (+ 2499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:15.874443 (+ 4922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:15.874455 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:15.874472 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:15.875328 (+ 856us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:15.875845 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:15.875849 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:15.875852 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:15.876071 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:15.876857 (+ 786us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:15.876864 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:15.877283 (+ 419us) client_negotiation.cc:769] Sending connection context
0807 18:15:15.877564 (+ 281us) client_negotiation.cc:240] Negotiation successful
0807 18:15:15.878042 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":420,"threads_started":1}
I20260807 18:15:15.881282 30047 heartbeater.cc:347] Connected to a master server at 127.17.110.62:38611
I20260807 18:15:15.881672 30047 heartbeater.cc:464] Registering TS with master...
I20260807 18:15:15.882864 30047 heartbeater.cc:511] Master 127.17.110.62:38611 requested a full tablet report, sending...
I20260807 18:15:15.884756 29558 ts_manager.cc:194] Registered new tserver with Master: 9b23c0f33641401193a33a09fd13b218 (127.17.110.2:39667)
I20260807 18:15:15.885746 29558 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.2@KRBTEST.COM'} at 127.17.110.2:51707
WARNING: no policy specified for kudu/127.17.110.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
I20260807 18:15:15.929563 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.3:0
--local_ip_for_outbound_sockets=127.17.110.3
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:38611
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:35207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:15:16.299099 29849 heartbeater.cc:503] Master 127.17.110.62:38611 was elected leader, sending a full tablet report...
W20260807 18:15:16.325438 30057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:16.325851 30057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:16.326026 30057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:16.334661 30057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:16.334841 30057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:16.335058 30057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.3
I20260807 18:15:16.345953 30057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:35207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:38611
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.3
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:15:16.347476 30057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:16.348872 30057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:16.364265 30063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:16.364339 30068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:16.364286 30064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:16.365227 30057 server_base.cc:1034] running on GCE node
I20260807 18:15:16.365917 30057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:16.367079 30057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:16.368330 30057 hybrid_clock.cc:648] HybridClock initialized: now 1786126516368288 us; error 39 us; skew 500 ppm
I20260807 18:15:16.368696 30057 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:15:16 dist-test-slave-39l8 krb5kdc[29494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126516, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:16.372406 30057 init.cc:377] Logged in from keytab as kudu/127.17.110.3@KRBTEST.COM (short username kudu)
I20260807 18:15:16.374169 30057 webserver.cc:460] Webserver started at http://127.17.110.3:42157/ using document root <none> and password file <none>
I20260807 18:15:16.375079 30057 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:16.375234 30057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:16.375589 30057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:15:16.378942 30057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-2/data/instance:
uuid: "706553ecbfe546799fd52daf6b7a069f"
format_stamp: "Formatted at 2026-08-07 18:15:16 on dist-test-slave-39l8"
server_key: "1f507b95520a6c73ee6d3afe939164b4"
server_key_iv: "ffc0cff2d28d1c6b2b86cf9e2be997db"
server_key_version: "encryptionkey@0"
I20260807 18:15:16.379978 30057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance:
uuid: "706553ecbfe546799fd52daf6b7a069f"
format_stamp: "Formatted at 2026-08-07 18:15:16 on dist-test-slave-39l8"
server_key: "1f507b95520a6c73ee6d3afe939164b4"
server_key_iv: "ffc0cff2d28d1c6b2b86cf9e2be997db"
server_key_version: "encryptionkey@0"
I20260807 18:15:16.386137 30057 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 18:15:16.390484 30077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:16.392107 30057 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:15:16.392325 30057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-2/wal
uuid: "706553ecbfe546799fd52daf6b7a069f"
format_stamp: "Formatted at 2026-08-07 18:15:16 on dist-test-slave-39l8"
server_key: "1f507b95520a6c73ee6d3afe939164b4"
server_key_iv: "ffc0cff2d28d1c6b2b86cf9e2be997db"
server_key_version: "encryptionkey@0"
I20260807 18:15:16.392560 30057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:16.408068 30057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:16.414036 30057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:16.414438 30057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:16.415699 30057 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:15:16.417447 30057 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:15:16.417567 30057 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:16.417732 30057 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:15:16.417810 30057 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:16.459950 30057 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.3:33189
I20260807 18:15:16.460002 30239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.3:33189 every 8 connection(s)
I20260807 18:15:16.461463 30057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
I20260807 18:15:16.471832 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 30057
I20260807 18:15:16.472034 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance
I20260807 18:15:16.472575 17848 external_mini_cluster.cc:1428] Setting key 357a51bf78204659c44710d4b9bb4e9e
Aug 07 18:15:16 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126516, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:15:16.485112 30242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:16.464594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:58361 (local address 127.17.110.62:38611)
0807 18:15:16.465140 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:16.465145 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:16.466018 (+ 873us) server_negotiation.cc:407] Connection header received
0807 18:15:16.468199 (+ 2181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:16.468205 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:16.468307 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:16.468432 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:16.470160 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:16.471084 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:16.473032 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:16.473615 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:16.473772 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:16.478276 (+ 4504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:16.478282 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:16.478286 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:16.478326 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:16.481315 (+ 2989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:16.482299 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:16.482304 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:16.482306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:16.482494 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:16.483118 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:16.483122 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:16.483124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:16.483469 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:16.483684 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:16.484596 (+ 912us) server_negotiation.cc:299] Negotiation successful
0807 18:15:16.484818 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":289,"threads_started":1}
I20260807 18:15:16.486312 30244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:16.464945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.3:58361)
0807 18:15:16.465754 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:16.465795 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:16.467870 (+ 2075us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:16.468701 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:16.468711 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:16.469322 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:16.469949 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:16.469969 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:16.471265 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:16.471270 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:16.472855 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:16.472871 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:16.473887 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:16.473896 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:16.474258 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:16.475214 (+ 956us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:16.475246 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:16.477997 (+ 2751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:16.481514 (+ 3517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:16.481523 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:16.481540 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:16.482140 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:16.482633 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:16.482640 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:16.482644 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:16.482979 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:16.483679 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:16.483689 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:16.484114 (+ 425us) client_negotiation.cc:769] Sending connection context
0807 18:15:16.484470 (+ 356us) client_negotiation.cc:240] Negotiation successful
0807 18:15:16.484945 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":288,"threads_started":1}
I20260807 18:15:16.488401 30240 heartbeater.cc:347] Connected to a master server at 127.17.110.62:38611
I20260807 18:15:16.488857 30240 heartbeater.cc:464] Registering TS with master...
I20260807 18:15:16.489854 30240 heartbeater.cc:511] Master 127.17.110.62:38611 requested a full tablet report, sending...
I20260807 18:15:16.491741 29557 ts_manager.cc:194] Registered new tserver with Master: 706553ecbfe546799fd52daf6b7a069f (127.17.110.3:33189)
I20260807 18:15:16.492746 29557 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.3@KRBTEST.COM'} at 127.17.110.3:58361
I20260807 18:15:16.502516 17848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.17.110.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.110.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.110.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.1.keytab.
Entry for principal oryx/127.17.110.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.1.keytab.
WARNING: no policy specified for oryx/127.17.110.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.110.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.110.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.2.keytab.
Entry for principal oryx/127.17.110.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.2.keytab.
WARNING: no policy specified for oryx/127.17.110.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.110.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.110.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.3.keytab.
Entry for principal oryx/127.17.110.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.3.keytab.
I20260807 18:15:16.695103 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 29652
Aug 07 18:15:16 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.1@KRBTEST.COM
I20260807 18:15:16.714844 30255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:16.696497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49208 (local address 127.17.110.1:38351)
0807 18:15:16.697229 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:16.697237 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:16.697273 (+ 36us) server_negotiation.cc:407] Connection header received
0807 18:15:16.697340 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:16.697347 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:16.697635 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:16.697802 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:16.699539 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:16.700742 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:16.702340 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:16.703118 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:16.703250 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:16.708072 (+ 4822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:16.708090 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:16.708107 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:16.708156 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:16.711441 (+ 3285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:16.712339 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:16.712349 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:16.712351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:16.712517 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:16.712999 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:16.713003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:16.713006 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:16.713564 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:16.713847 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:16.714224 (+ 377us) server_negotiation.cc:299] Negotiation successful
0807 18:15:16.714456 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":392,"threads_started":1}
I20260807 18:15:16.716279 29844 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:16.804991 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 29652
I20260807 18:15:16.830475 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 29859
Aug 07 18:15:16 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.2@KRBTEST.COM
I20260807 18:15:16.848464 30258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:16.831658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34738 (local address 127.17.110.2:39667)
0807 18:15:16.832220 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:16.832227 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:16.832253 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:15:16.832314 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:16.832320 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:16.832518 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:16.832661 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:16.833936 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:16.834985 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:16.836310 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:16.836853 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:16.836954 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:16.841583 (+ 4629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:16.841599 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:16.841617 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:16.841850 (+ 233us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:16.845300 (+ 3450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:16.846089 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:16.846097 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:16.846100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:16.846243 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:16.846658 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:16.846664 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:16.846667 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:16.847106 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:16.847441 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:16.847772 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 18:15:16.848102 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":278,"threads_started":1}
I20260807 18:15:16.849861 30039 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:16.934286 30047 heartbeater.cc:503] Master 127.17.110.62:38611 was elected leader, sending a full tablet report...
I20260807 18:15:16.935276 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 29859
W20260807 18:15:16.956540 29534 connection.cc:535] server connection from 127.17.110.2:51707 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 18:15:16.956864 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 30057
Aug 07 18:15:16 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.3@KRBTEST.COM
I20260807 18:15:16.974694 30261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:16.957882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42720 (local address 127.17.110.3:33189)
0807 18:15:16.958633 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:16.958641 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:16.958675 (+ 34us) server_negotiation.cc:407] Connection header received
0807 18:15:16.958806 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:16.958813 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:16.959096 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:16.959289 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:16.960640 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:16.961640 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:16.962974 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:16.963640 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:16.963795 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:16.968585 (+ 4790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:16.968599 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:16.968616 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:16.968667 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:16.971811 (+ 3144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:16.972613 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:16.972622 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:16.972625 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:16.972769 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:16.973135 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:16.973139 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:16.973141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:16.973551 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:16.973829 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:16.974123 (+ 294us) server_negotiation.cc:299] Negotiation successful
0807 18:15:16.974348 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":431,"threads_started":1}
I20260807 18:15:16.976279 30234 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:17.061870 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 30057
I20260807 18:15:17.088423 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 29512
I20260807 18:15:17.089883 29605 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:17.196708 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 29512
I20260807 18:15:17.228641 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:38611
--webserver_interface=127.17.110.62
--webserver_port=46173
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:35207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:38611
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:15:17.589221 30267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:17.589560 30267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:17.589708 30267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:17.597316 30267 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:15:17.597452 30267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:17.597538 30267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:17.597627 30267 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:15:17.597734 30267 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:15:17.608228 30267 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:35207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:38611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:38611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=46173
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:15:17.609720 30267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:17.611162 30267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:17.624872 30278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:17.624933 30275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:17.624936 30276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:17.625917 30267 server_base.cc:1034] running on GCE node
I20260807 18:15:17.626968 30267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:17.628618 30267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:17.629889 30267 hybrid_clock.cc:648] HybridClock initialized: now 1786126517629822 us; error 52 us; skew 500 ppm
I20260807 18:15:17.630316 30267 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:15:17 dist-test-slave-39l8 krb5kdc[29494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126517, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:17.633991 30267 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:15:17.635871 30267 webserver.cc:460] Webserver started at http://127.17.110.62:46173/ using document root <none> and password file <none>
I20260807 18:15:17.636706 30267 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:17.636878 30267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:17.643219 30267 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:15:17.646636 30290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:17.648548 30267 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:15:17.648823 30267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "dce360763d52470b9281fe028082ac03"
format_stamp: "Formatted at 2026-08-07 18:15:14 on dist-test-slave-39l8"
server_key: "85f3513ce8951b9753252e23f14991c5"
server_key_iv: "822b50fd9b983b35ffdcdeee2a28d387"
server_key_version: "encryptionkey@0"
I20260807 18:15:17.650066 30267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:17.668632 30267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:17.673677 30267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:17.674003 30267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:17.695070 30267 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:38611
I20260807 18:15:17.695089 30366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:38611 every 8 connection(s)
I20260807 18:15:17.696787 30267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:15:17.698427 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 30267
I20260807 18:15:17.699570 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.1:38351
--local_ip_for_outbound_sockets=127.17.110.1
--tserver_master_addrs=127.17.110.62:38611
--webserver_port=42059
--webserver_interface=127.17.110.1
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:35207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.110.1
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:15:17.707670 30367 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:15:17.712535 30367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03: Bootstrap starting.
I20260807 18:15:17.733244 30367 log.cc:824] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03: Log is configured to *not* fsync() on all Append() calls
I20260807 18:15:17.743121 30367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:15:17.743736 30367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03: Bootstrap complete.
I20260807 18:15:17.753338 30367 raft_consensus.cc:348] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dce360763d52470b9281fe028082ac03" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38611 } }
I20260807 18:15:17.754256 30367 raft_consensus.cc:729] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dce360763d52470b9281fe028082ac03, State: Initialized, Role: FOLLOWER
I20260807 18:15:17.755066 30367 consensus_queue.cc:260] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dce360763d52470b9281fe028082ac03" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38611 } }
I20260807 18:15:17.755322 30367 raft_consensus.cc:388] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:15:17.755445 30367 raft_consensus.cc:482] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:15:17.755656 30367 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:15:17.759944 30367 raft_consensus.cc:504] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dce360763d52470b9281fe028082ac03" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38611 } }
I20260807 18:15:17.760566 30367 leader_election.cc:302] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dce360763d52470b9281fe028082ac03; no voters:
I20260807 18:15:17.761265 30367 leader_election.cc:288] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 18:15:17.761420 30372 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 18:15:17.762732 30372 raft_consensus.cc:686] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [term 2 LEADER]: Becoming Leader. State: Replica: dce360763d52470b9281fe028082ac03, State: Running, Role: LEADER
I20260807 18:15:17.763351 30372 consensus_queue.cc:237] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dce360763d52470b9281fe028082ac03" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38611 } }
I20260807 18:15:17.763713 30367 sys_catalog.cc:564] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:15:17.766000 30374 sys_catalog.cc:455] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dce360763d52470b9281fe028082ac03. Latest consensus state: current_term: 2 leader_uuid: "dce360763d52470b9281fe028082ac03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dce360763d52470b9281fe028082ac03" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38611 } } }
I20260807 18:15:17.766402 30374 sys_catalog.cc:458] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [sys.catalog]: This master's current role is: LEADER
I20260807 18:15:17.766992 30373 sys_catalog.cc:455] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "dce360763d52470b9281fe028082ac03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dce360763d52470b9281fe028082ac03" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38611 } } }
I20260807 18:15:17.767302 30373 sys_catalog.cc:458] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03 [sys.catalog]: This master's current role is: LEADER
I20260807 18:15:17.775890 30381 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:15:17.783418 30381 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:15:17.785408 30381 catalog_manager.cc:1259] Loaded cluster ID: a0ecce4b3b394f5389bd177db17259af
I20260807 18:15:17.785563 30381 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:15:17.791301 30381 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:15:17.793497 30381 catalog_manager.cc:5879] T 00000000000000000000000000000000 P dce360763d52470b9281fe028082ac03: Loaded TSK: 0
I20260807 18:15:17.795742 30381 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 18:15:18.091985 30369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:18.092334 30369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:18.092490 30369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:18.101477 30369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:18.101643 30369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:18.101858 30369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.1
I20260807 18:15:18.114485 30369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:35207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.1.keytab
--principal=oryx/127.17.110.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.1:38351
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.110.1
--webserver_port=42059
--tserver_master_addrs=127.17.110.62:38611
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.1
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:15:18.116000 30369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:18.117542 30369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:18.132288 30412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:18.132344 30411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:18.133174 30420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:18.133538 30369 server_base.cc:1034] running on GCE node
I20260807 18:15:18.134310 30369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:18.135505 30369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:18.136772 30369 hybrid_clock.cc:648] HybridClock initialized: now 1786126518136698 us; error 60 us; skew 500 ppm
I20260807 18:15:18.137132 30369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:15:18 dist-test-slave-39l8 krb5kdc[29494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126518, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:18.140887 30369 init.cc:377] Logged in from keytab as oryx/127.17.110.1@KRBTEST.COM (short username oryx)
I20260807 18:15:18.142699 30369 webserver.cc:460] Webserver started at http://127.17.110.1:42059/ using document root <none> and password file <none>
I20260807 18:15:18.143522 30369 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:18.143700 30369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:18.149863 30369 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:15:18.153136 30434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:18.155314 30369 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:15:18.155577 30369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-0/wal
uuid: "c74dbc44bf1b484bb643820f64c42cad"
format_stamp: "Formatted at 2026-08-07 18:15:15 on dist-test-slave-39l8"
server_key: "6b4517793d3ae2a57e9e10616d47d443"
server_key_iv: "1c82d5bac67b92564ee0a7a61d55ce1a"
server_key_version: "encryptionkey@0"
I20260807 18:15:18.156796 30369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:18.186333 30369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:18.192171 30369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:18.192556 30369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:18.193863 30369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:15:18.195679 30369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:15:18.195791 30369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:18.195876 30369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:15:18.195942 30369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:18.234637 30369 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.1:38351
I20260807 18:15:18.234747 30597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.1:38351 every 8 connection(s)
I20260807 18:15:18.236657 30369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
I20260807 18:15:18.242322 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 30369
I20260807 18:15:18.243057 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.2:39667
--local_ip_for_outbound_sockets=127.17.110.2
--tserver_master_addrs=127.17.110.62:38611
--webserver_port=40267
--webserver_interface=127.17.110.2
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:35207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.110.2
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:15:18 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:18 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 18:15:18.649333 30607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:18.649659 30607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:18.649811 30607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:18.658036 30607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:18.658175 30607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:18.658396 30607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.2
I20260807 18:15:18.669313 30607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:35207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.2.keytab
--principal=oryx/127.17.110.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.2:39667
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.110.2
--webserver_port=40267
--tserver_master_addrs=127.17.110.62:38611
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.2
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:15:18.670888 30607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:18.672298 30607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:18.688073 30616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:18.688313 30618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:18.688982 30621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:18.694382 30607 server_base.cc:1034] running on GCE node
I20260807 18:15:18.695760 30607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:18.697655 30607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:18.699025 30607 hybrid_clock.cc:648] HybridClock initialized: now 1786126518698930 us; error 67 us; skew 500 ppm
I20260807 18:15:18.699645 30607 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:15:18 dist-test-slave-39l8 krb5kdc[29494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126518, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:18.706012 30607 init.cc:377] Logged in from keytab as oryx/127.17.110.2@KRBTEST.COM (short username oryx)
I20260807 18:15:18.708851 30607 webserver.cc:460] Webserver started at http://127.17.110.2:40267/ using document root <none> and password file <none>
I20260807 18:15:18.710352 30607 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:18.710603 30607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:18.721719 30607 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.005s sys 0.000s
I20260807 18:15:18.727005 30629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:18.729575 30607 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.002s
I20260807 18:15:18.729903 30607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-1/wal
uuid: "9b23c0f33641401193a33a09fd13b218"
format_stamp: "Formatted at 2026-08-07 18:15:15 on dist-test-slave-39l8"
server_key: "6804726fa4c7a28351e7c4729133bb73"
server_key_iv: "c82886019cf74418dd5829b40da2d9db"
server_key_version: "encryptionkey@0"
I20260807 18:15:18.732268 30607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:18.766400 30607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:18.774609 30607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:18.775163 30607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:18.777119 30607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:15:18.779918 30607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:15:18.780102 30607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:18.780269 30607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:15:18.780385 30607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:18.824002 30786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.2:39667 every 8 connection(s)
I20260807 18:15:18.824018 30607 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.2:39667
I20260807 18:15:18.827061 30607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
I20260807 18:15:18.827822 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 30607
I20260807 18:15:18.828668 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.3:33189
--local_ip_for_outbound_sockets=127.17.110.3
--tserver_master_addrs=127.17.110.62:38611
--webserver_port=42157
--webserver_interface=127.17.110.3
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:35207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.110.3
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:15:18 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:18 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 18:15:19.259433 30795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:19.259821 30795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:19.260012 30795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:19.268193 30795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:19.268378 30795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:19.268584 30795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.3
I20260807 18:15:19.279328 30795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:35207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/krb5kdc/oryx_127.17.110.3.keytab
--principal=oryx/127.17.110.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.3:33189
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.110.3
--webserver_port=42157
--tserver_master_addrs=127.17.110.62:38611
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.3
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:15:19.280890 30795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:19.282420 30795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:19.301193 30805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:19.301299 30803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:19.308768 30807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:19.309799 30795 server_base.cc:1034] running on GCE node
I20260807 18:15:19.310676 30795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:19.311973 30795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:19.313239 30795 hybrid_clock.cc:648] HybridClock initialized: now 1786126519313167 us; error 77 us; skew 500 ppm
I20260807 18:15:19.313616 30795 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:15:19 dist-test-slave-39l8 krb5kdc[29494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126519, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.110.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:19.317276 30795 init.cc:377] Logged in from keytab as oryx/127.17.110.3@KRBTEST.COM (short username oryx)
I20260807 18:15:19.319423 30795 webserver.cc:460] Webserver started at http://127.17.110.3:42157/ using document root <none> and password file <none>
I20260807 18:15:19.320250 30795 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:19.320436 30795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:19.326608 30795 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:15:19.329994 30818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:19.331722 30795 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260807 18:15:19.331940 30795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-2/wal
uuid: "706553ecbfe546799fd52daf6b7a069f"
format_stamp: "Formatted at 2026-08-07 18:15:16 on dist-test-slave-39l8"
server_key: "1f507b95520a6c73ee6d3afe939164b4"
server_key_iv: "ffc0cff2d28d1c6b2b86cf9e2be997db"
server_key_version: "encryptionkey@0"
I20260807 18:15:19.333113 30795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:19.358240 30795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:19.363914 30795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:19.364259 30795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:19.366554 30795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:15:19.371616 30795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:15:19.371783 30795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:19.371948 30795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:15:19.372061 30795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:19.418673 30795 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.3:33189
I20260807 18:15:19.418741 30993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.3:33189 every 8 connection(s)
I20260807 18:15:19.420388 30795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
I20260807 18:15:19.428625 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 30795
Aug 07 18:15:19 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:19 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:19.464105 31004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:19.431060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50180 (local address 127.17.110.62:38611)
0807 18:15:19.441808 (+ 10748us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:19.441816 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:19.441837 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:15:19.441904 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:19.441912 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:19.442005 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:19.442140 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:19.445576 (+ 3436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:19.446998 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:19.448995 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:19.449669 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:19.449767 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:19.453971 (+ 4204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:19.453984 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:19.454010 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:19.454069 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:19.458927 (+ 4858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:19.459971 (+ 1044us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:19.459981 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:19.459990 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:19.460198 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:19.460660 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:19.460666 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:19.460671 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:19.461284 (+ 613us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:19.461686 (+ 402us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:19.462176 (+ 490us) server_negotiation.cc:299] Negotiation successful
0807 18:15:19.462782 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10611,"thread_start_us":10421,"threads_started":1}
W20260807 18:15:19.818521 30606 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.567s user 0.000s sys 0.017s
W20260807 18:15:19.820956 30600 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:18.239951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:33851 (local address 127.17.110.62:38611)
0807 18:15:18.240792 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:18.240805 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:18.241113 (+ 308us) server_negotiation.cc:407] Connection header received
0807 18:15:18.243563 (+ 2450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:18.243576 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:18.243945 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:18.244334 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:18.245970 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:18.247341 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:18.249051 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:18.249738 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:18.249961 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:19.820439 (+1570478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.1:33851: BlockingRecv error: recv got EOF from 127.17.110.1:33851 (error 108)
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":375,"threads_started":1}
I20260807 18:15:19.823313 30606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:18.240065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.1:33851)
0807 18:15:18.240920 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:18.240970 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:18.242964 (+ 1994us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:18.244488 (+ 1524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:18.244502 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:18.245163 (+ 661us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:18.245725 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:18.245746 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:18.247538 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:18.247543 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:18.248884 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:18.248899 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:18.250000 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:18.250008 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:18.250313 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:18.251161 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:18.251187 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:19.821489 (+1570302us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":372,"threads_started":1}
W20260807 18:15:19.825197 30598 heartbeater.cc:644] Failed to heartbeat to 127.17.110.62:38611 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.110.62:38611: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Aug 07 18:15:19 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:19 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:19.862655 30606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:19.826269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.1:37867)
0807 18:15:19.830596 (+ 4327us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:19.830632 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:19.831930 (+ 1298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:19.833083 (+ 1153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:19.833093 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:19.833841 (+ 748us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:19.834178 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:19.834197 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:19.838617 (+ 4420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:19.838628 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:19.843321 (+ 4693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:19.843357 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:19.846384 (+ 3027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:19.846397 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:19.846780 (+ 383us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:19.853472 (+ 6692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:19.853517 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:19.861907 (+ 8390us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":4117,"thread_start_us":21157,"threads_started":1}
W20260807 18:15:19.862919 30600 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:19.827403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:37867 (local address 127.17.110.62:38611)
0807 18:15:19.827699 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:19.827709 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:19.830819 (+ 3110us) server_negotiation.cc:407] Connection header received
0807 18:15:19.832431 (+ 1612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:19.832441 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:19.832569 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:19.832690 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:19.835909 (+ 3219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:19.838194 (+ 2285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:19.844805 (+ 6611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:19.845857 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:19.846018 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:19.862349 (+ 16331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.1:37867: BlockingRecv error: recv got EOF from 127.17.110.1:37867 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 07 18:15:19 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:19 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:19.889025 30606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:19.864781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.1:35585)
0807 18:15:19.865144 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:19.865174 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:19.865366 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:19.867234 (+ 1868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:19.867244 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:19.868028 (+ 784us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:19.868427 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:19.868447 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:19.871235 (+ 2788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:19.871248 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:19.874057 (+ 2809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:19.874095 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:19.876085 (+ 1990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:19.876099 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:19.876440 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:19.877657 (+ 1217us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:19.877703 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:19.888546 (+ 10843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191}
W20260807 18:15:19.889595 30598 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:15:19.889871 30600 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:19.864787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:35585 (local address 127.17.110.62:38611)
0807 18:15:19.865187 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:19.865198 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:19.865266 (+ 68us) server_negotiation.cc:407] Connection header received
0807 18:15:19.866132 (+ 866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:19.866143 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:19.866291 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:19.866472 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:19.868745 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:19.870916 (+ 2171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:19.874438 (+ 3522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:19.875659 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:19.876051 (+ 392us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:19.889016 (+ 12965us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.1:35585: BlockingRecv error: recv got EOF from 127.17.110.1:35585 (error 108)
Metrics: {"server-negotiator.queue_time_us":194}
W20260807 18:15:20.347229 30792 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.489s user 0.007s sys 0.015s
W20260807 18:15:20.348819 30794 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:18.832158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:43629 (local address 127.17.110.62:38611)
0807 18:15:18.841676 (+ 9518us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:18.841684 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:18.842660 (+ 976us) server_negotiation.cc:407] Connection header received
0807 18:15:18.845327 (+ 2667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:18.845333 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:18.845446 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:18.845574 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:18.849118 (+ 3544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:18.850458 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:18.854613 (+ 4155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:18.855559 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:18.855756 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:20.348490 (+1492734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.2:43629: BlockingRecv error: recv got EOF from 127.17.110.2:43629 (error 108)
Metrics: {"server-negotiator.queue_time_us":9316,"thread_start_us":9702,"threads_started":1}
I20260807 18:15:20.349355 30792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:18.831426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.2:43629)
0807 18:15:18.842404 (+ 10978us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:18.842461 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:18.844985 (+ 2524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:18.846101 (+ 1116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:18.846122 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:18.847389 (+ 1267us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:18.848727 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:18.848772 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:18.850778 (+ 2006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:18.850789 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:18.854270 (+ 3481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:18.854312 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:18.856047 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:18.856062 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:18.856588 (+ 526us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:18.857916 (+ 1328us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:18.857969 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:20.348197 (+1490228us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":9491,"thread_start_us":9351,"threads_started":1}
W20260807 18:15:20.350339 30787 heartbeater.cc:644] Failed to heartbeat to 127.17.110.62:38611 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.110.62:38611: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Aug 07 18:15:20 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:20 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:20.361413 30792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:20.351221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.2:55313)
0807 18:15:20.351452 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:20.351472 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:20.351626 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:20.351991 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:20.351996 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:20.352409 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:20.352583 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:20.352592 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:20.353616 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:20.353622 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:20.354779 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:20.354801 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:20.355647 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:20.355653 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:20.355842 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:20.356510 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:20.356528 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:20.361078 (+ 4550us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99}
W20260807 18:15:20.361578 30794 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:20.351222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:55313 (local address 127.17.110.62:38611)
0807 18:15:20.351405 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:20.351411 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:20.351554 (+ 143us) server_negotiation.cc:407] Connection header received
0807 18:15:20.351789 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:20.351793 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:20.351857 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:20.351960 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:20.352736 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:20.353454 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:20.354931 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:20.355460 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:20.355616 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:20.361342 (+ 5726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.2:55313: BlockingRecv error: recv got EOF from 127.17.110.2:55313 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 07 18:15:20 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:20 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:20.374400 30792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:20.362187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.2:57287)
0807 18:15:20.362358 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:20.362379 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:20.362494 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:20.363213 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:20.363230 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:20.363566 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:20.363726 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:20.363735 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:20.364872 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:20.364878 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:20.366134 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:20.366150 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:20.367423 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:20.367432 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:20.367692 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:20.368505 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:20.368545 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:20.374104 (+ 5559us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 18:15:20.374809 30787 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:15:20.375499 30794 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:20.362752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:57287 (local address 127.17.110.62:38611)
0807 18:15:20.362924 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:20.362929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:20.362942 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:15:20.362986 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:20.362990 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:20.363059 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:20.363171 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:20.363873 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:20.364715 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:20.366322 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:20.367220 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:20.367353 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:20.375231 (+ 7878us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.2:57287: BlockingRecv error: recv got EOF from 127.17.110.2:57287 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260807 18:15:20.645551 31000 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.190s user 0.004s sys 0.014s
W20260807 18:15:20.647167 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:19.425032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:43629 (local address 127.17.110.62:38611)
0807 18:15:19.435411 (+ 10379us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:19.435423 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:19.441296 (+ 5873us) server_negotiation.cc:407] Connection header received
0807 18:15:19.445387 (+ 4091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:19.445398 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:19.445541 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:19.445626 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:19.448538 (+ 2912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:19.449827 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:19.452698 (+ 2871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:19.453453 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:19.453570 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:20.646813 (+1193243us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.3:43629: BlockingRecv error: recv got EOF from 127.17.110.3:43629 (error 108)
Metrics: {"server-negotiator.queue_time_us":10135,"thread_start_us":10606,"threads_started":1}
I20260807 18:15:20.648000 31000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:19.425164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.3:43629)
0807 18:15:19.440984 (+ 15820us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:19.441101 (+ 117us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:19.444754 (+ 3653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:19.446428 (+ 1674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:19.446443 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:19.447357 (+ 914us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:19.448288 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:19.448317 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:19.450054 (+ 1737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:19.450059 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:19.452463 (+ 2404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:19.452487 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:19.454011 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:19.454018 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:19.454395 (+ 377us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:19.455306 (+ 911us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:19.455358 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:20.646587 (+1191229us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":15277,"thread_start_us":14988,"threads_started":1}
W20260807 18:15:20.649116 30994 heartbeater.cc:644] Failed to heartbeat to 127.17.110.62:38611 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.110.62:38611: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Aug 07 18:15:20 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:20 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:20.659991 31000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:20.649841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.3:55623)
0807 18:15:20.650057 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:20.650075 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:20.650179 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:20.650519 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:20.650523 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:20.650985 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:20.651171 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:20.651180 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:20.652396 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:20.652400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:20.653433 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:20.653447 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:20.654306 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:20.654310 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:20.654435 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:20.655173 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:20.655192 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:20.659617 (+ 4425us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":72}
W20260807 18:15:20.660142 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:20.649968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:55623 (local address 127.17.110.62:38611)
0807 18:15:20.650152 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:20.650157 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:20.650176 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:15:20.650317 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:20.650321 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:20.650388 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:20.650467 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:20.651412 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:20.652236 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:20.653597 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:20.654069 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:20.654252 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:20.659824 (+ 5572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.3:55623: BlockingRecv error: recv got EOF from 127.17.110.3:55623 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 07 18:15:20 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:20 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:20.671181 31000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:20.660959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.3:52285)
0807 18:15:20.661280 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:20.661300 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:20.661410 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:20.661905 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:20.661909 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:20.662339 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:20.662523 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:20.662532 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:20.663883 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:20.663889 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:20.665136 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:20.665154 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:20.666026 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:20.666031 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:20.666204 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:20.666941 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:20.666962 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:20.670865 (+ 3903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167}
W20260807 18:15:20.671439 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:20.661320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:52285 (local address 127.17.110.62:38611)
0807 18:15:20.661595 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:20.661600 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:20.661618 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:15:20.661669 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:20.661673 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:20.661753 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:20.661843 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:20.662680 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:20.663647 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:20.665328 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:20.665852 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:20.665970 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:20.671121 (+ 5151us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.3:52285: BlockingRecv error: recv got EOF from 127.17.110.3:52285 (error 108)
Metrics: {"server-negotiator.queue_time_us":149}
W20260807 18:15:20.671774 30994 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 18:15:20 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:20 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:20.901172 31007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:20.890294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.1:36719)
0807 18:15:20.890921 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:20.890945 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:20.891097 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:20.891512 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:20.891516 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:20.891938 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:20.892101 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:20.892110 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:20.893354 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:20.893358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:20.894312 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:20.894324 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:20.895293 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:20.895299 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:20.895439 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:20.896158 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:20.896178 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:20.900901 (+ 4723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":333,"threads_started":1}
W20260807 18:15:20.901600 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:20.890353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:36719 (local address 127.17.110.62:38611)
0807 18:15:20.890589 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:20.890595 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:20.891039 (+ 444us) server_negotiation.cc:407] Connection header received
0807 18:15:20.891250 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:20.891256 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:20.891343 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:20.891451 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:20.892268 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:20.893205 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:20.894477 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:20.895080 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:20.895199 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:20.901273 (+ 6074us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.1:36719: BlockingRecv error: recv got EOF from 127.17.110.1:36719 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 07 18:15:21 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:21 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:21.385948 31009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:21.375527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.2:36575)
0807 18:15:21.376120 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:21.376136 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:21.376312 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:21.376714 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:21.376718 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:21.377218 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:21.377379 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:21.377389 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:21.378515 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:21.378520 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:21.379582 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:21.379595 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:21.380656 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:21.380663 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:21.380847 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:21.381535 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:21.381554 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:21.385676 (+ 4122us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":349,"threads_started":1}
W20260807 18:15:21.386164 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:21.375662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:36575 (local address 127.17.110.62:38611)
0807 18:15:21.375925 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:21.375934 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:21.376231 (+ 297us) server_negotiation.cc:407] Connection header received
0807 18:15:21.376477 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:21.376484 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:21.376565 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:21.376677 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:21.377577 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:21.378358 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:21.379738 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:21.380442 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:21.380592 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:21.385910 (+ 5318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.2:36575: BlockingRecv error: recv got EOF from 127.17.110.2:36575 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 07 18:15:21 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:21 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:21.683614 31010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:21.672560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.3:38677)
0807 18:15:21.673285 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:21.673305 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:21.673445 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:21.673838 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:21.673843 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:21.674337 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:21.674509 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:21.674519 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:21.675719 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:21.675725 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:21.676824 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:21.676839 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:21.677912 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:21.677918 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:21.678054 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:21.678793 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:21.678816 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:21.683320 (+ 4504us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":443,"threads_started":1}
W20260807 18:15:21.683923 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:21.672730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:38677 (local address 127.17.110.62:38611)
0807 18:15:21.672918 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:21.672923 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:21.673456 (+ 533us) server_negotiation.cc:407] Connection header received
0807 18:15:21.673556 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:21.673560 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:21.673630 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:21.673730 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:21.674672 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:21.675563 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:21.676995 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:21.677649 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:21.677845 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:21.683609 (+ 5764us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.3:38677: BlockingRecv error: recv got EOF from 127.17.110.3:38677 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 07 18:15:21 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:21 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:21.913542 31012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:21.902383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.1:50037)
0807 18:15:21.903048 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:21.903066 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:21.903242 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:21.903747 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:21.903754 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:21.904390 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:21.904606 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:21.904618 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:21.905773 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:21.905778 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:21.906912 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:21.906928 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:21.907720 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:21.907726 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:21.907865 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:21.908571 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:21.908595 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:21.913213 (+ 4618us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":412,"threads_started":1}
W20260807 18:15:21.913923 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:21.902511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:50037 (local address 127.17.110.62:38611)
0807 18:15:21.902791 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:21.902798 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:21.903166 (+ 368us) server_negotiation.cc:407] Connection header received
0807 18:15:21.903400 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:21.903404 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:21.903520 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:21.903631 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:21.904776 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:21.905586 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:21.907086 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:21.907515 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:21.907633 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:21.913614 (+ 5981us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.1:50037: BlockingRecv error: recv got EOF from 127.17.110.1:50037 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 07 18:15:22 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:22 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:22.398067 31013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:22.387203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.2:37277)
0807 18:15:22.387795 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:22.387813 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:22.387986 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:22.388379 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:22.388384 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:22.388899 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:22.389105 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:22.389116 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:22.390292 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:22.390304 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:22.391404 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:22.391418 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:22.392258 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:22.392265 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:22.392464 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:22.393131 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:22.393151 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:22.397784 (+ 4633us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":306,"threads_started":1}
W20260807 18:15:22.398237 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:22.387308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:37277 (local address 127.17.110.62:38611)
0807 18:15:22.387566 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:22.387576 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:22.387907 (+ 331us) server_negotiation.cc:407] Connection header received
0807 18:15:22.388129 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:22.388137 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:22.388216 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:22.388343 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:22.389336 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:22.390117 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:22.391564 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:22.392060 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:22.392223 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:22.398009 (+ 5786us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.2:37277: BlockingRecv error: recv got EOF from 127.17.110.2:37277 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 07 18:15:22 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:22 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:22.695792 31014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:22.684741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.3:34851)
0807 18:15:22.685494 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:22.685512 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:22.685706 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:22.686149 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:22.686156 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:22.686622 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:22.686918 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:22.686930 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:22.688035 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:22.688059 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:22.689001 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:22.689014 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:22.689783 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:22.689790 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:22.689989 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:22.690658 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:22.690678 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:22.695398 (+ 4720us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":501,"threads_started":1}
W20260807 18:15:22.695899 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:22.685100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:34851 (local address 127.17.110.62:38611)
0807 18:15:22.685370 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:22.685377 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:22.685607 (+ 230us) server_negotiation.cc:407] Connection header received
0807 18:15:22.685874 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:22.685880 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:22.685958 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:22.686073 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:22.687080 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:22.687884 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:22.689172 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:22.689598 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:22.689756 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:22.695640 (+ 5884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.3:34851: BlockingRecv error: recv got EOF from 127.17.110.3:34851 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Aug 07 18:15:22 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:22 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:22.926617 31015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:22.914813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.1:45407)
0807 18:15:22.915358 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:22.915374 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:22.915549 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:22.916069 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:22.916076 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:22.916533 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:22.916720 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:22.916730 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:22.917835 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:22.917842 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:22.918875 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:22.918890 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:22.919707 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:22.919714 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:22.919910 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:22.920648 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:22.920669 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:22.926196 (+ 5527us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":314,"threads_started":1}
W20260807 18:15:22.926903 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:22.914847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:45407 (local address 127.17.110.62:38611)
0807 18:15:22.915142 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:22.915151 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:22.915472 (+ 321us) server_negotiation.cc:407] Connection header received
0807 18:15:22.915825 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:22.915830 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:22.915911 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:22.916096 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:22.916888 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:22.917668 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:22.919052 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:22.919536 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:22.919685 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:22.926497 (+ 6812us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.1:45407: BlockingRecv error: recv got EOF from 127.17.110.1:45407 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 07 18:15:23 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:23 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:23.410053 31016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:23.399394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.2:36695)
0807 18:15:23.400088 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:23.400109 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:23.400246 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:23.400629 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:23.400633 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:23.401026 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:23.401250 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:23.401261 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:23.402323 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:23.402328 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:23.403331 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:23.403346 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:23.404209 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:23.404214 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:23.404367 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:23.405147 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:23.405168 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:23.409740 (+ 4572us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":382,"threads_started":1}
W20260807 18:15:23.410461 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:23.399519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:36695 (local address 127.17.110.62:38611)
0807 18:15:23.399793 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:23.399802 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:23.400224 (+ 422us) server_negotiation.cc:407] Connection header received
0807 18:15:23.400373 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:23.400377 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:23.400448 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:23.400539 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:23.401409 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:23.402157 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:23.403490 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:23.403959 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:23.404072 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:23.410136 (+ 6064us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.2:36695: BlockingRecv error: recv got EOF from 127.17.110.2:36695 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 07 18:15:23 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:23 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:23.707686 31017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:23.697179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.3:42625)
0807 18:15:23.697791 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:23.697813 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:23.697959 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:23.698366 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:23.698370 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:23.698945 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:23.699174 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:23.699186 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:23.700464 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:23.700468 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:23.701413 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:23.701426 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:23.702260 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:23.702266 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:23.702460 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:23.703176 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:23.703196 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:23.707318 (+ 4122us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":354,"threads_started":1}
W20260807 18:15:23.707759 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:23.697247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:42625 (local address 127.17.110.62:38611)
0807 18:15:23.697525 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:23.697537 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:23.697930 (+ 393us) server_negotiation.cc:407] Connection header received
0807 18:15:23.698143 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:23.698147 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:23.698214 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:23.698304 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:23.699370 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:23.700219 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:23.701588 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:23.702079 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:23.702245 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:23.707544 (+ 5299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.3:42625: BlockingRecv error: recv got EOF from 127.17.110.3:42625 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 07 18:15:23 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:23 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:23.938951 31018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:23.927874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.1:33695)
0807 18:15:23.928480 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:23.928498 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:23.928678 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:23.929084 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:23.929090 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:23.929575 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:23.929763 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:23.929773 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:23.931086 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:23.931091 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:23.932254 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:23.932269 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:23.933115 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:23.933122 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:23.933346 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:23.934069 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:23.934090 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:23.938557 (+ 4467us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":325,"threads_started":1}
W20260807 18:15:23.939182 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:23.928004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:33695 (local address 127.17.110.62:38611)
0807 18:15:23.928303 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:23.928311 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:23.928607 (+ 296us) server_negotiation.cc:407] Connection header received
0807 18:15:23.928847 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:23.928852 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:23.928924 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:23.929047 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:23.929959 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:23.930905 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:23.932437 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:23.932925 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:23.933089 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:23.938905 (+ 5816us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.1:33695: BlockingRecv error: recv got EOF from 127.17.110.1:33695 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Aug 07 18:15:24 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:24 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:24.421566 31019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:24.411212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.2:53525)
0807 18:15:24.411852 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:24.411869 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:24.412048 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:24.412468 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:24.412473 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:24.412921 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:24.413094 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:24.413103 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:24.414121 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:24.414129 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:24.415180 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:24.415193 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:24.415993 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:24.415999 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:24.416212 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:24.416870 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:24.416889 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:24.421182 (+ 4293us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":396,"threads_started":1}
W20260807 18:15:24.421674 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:24.411409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:53525 (local address 127.17.110.62:38611)
0807 18:15:24.411621 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:24.411629 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:24.411967 (+ 338us) server_negotiation.cc:407] Connection header received
0807 18:15:24.412237 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:24.412242 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:24.412309 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:24.412422 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:24.413249 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:24.413977 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:24.415335 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:24.415822 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:24.415965 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:24.421416 (+ 5451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.2:53525: BlockingRecv error: recv got EOF from 127.17.110.2:53525 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 07 18:15:24 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:24 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:24.719434 31021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:24.708860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.3:50305)
0807 18:15:24.709484 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:24.709503 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:24.709641 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:24.710015 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:24.710018 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:24.710501 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:24.710680 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:24.710691 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:24.711761 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:24.711765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:24.712922 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:24.712938 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:24.713948 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:24.713955 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:24.714238 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:24.715062 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:24.715082 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:24.719046 (+ 3964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":370,"threads_started":1}
W20260807 18:15:24.719595 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:24.709087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:50305 (local address 127.17.110.62:38611)
0807 18:15:24.709364 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:24.709371 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:24.709607 (+ 236us) server_negotiation.cc:407] Connection header received
0807 18:15:24.709792 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:24.709797 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:24.709872 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:24.709964 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:24.710866 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:24.711615 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:24.713125 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:24.713741 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:24.713888 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:24.719311 (+ 5423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.3:50305: BlockingRecv error: recv got EOF from 127.17.110.3:50305 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 07 18:15:24 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:24 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:24.955652 31024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:24.940386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.1:47845)
0807 18:15:24.940933 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:24.940951 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:24.941121 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:24.941560 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:24.941566 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:24.942129 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:24.942304 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:24.942314 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:24.943426 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:24.943431 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:24.944574 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:24.944588 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:24.945324 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:24.945331 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:24.945526 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:24.950906 (+ 5380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:24.950931 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:24.955381 (+ 4450us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":289,"threads_started":1}
W20260807 18:15:24.955969 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:24.940401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:47845 (local address 127.17.110.62:38611)
0807 18:15:24.940677 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:24.940688 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:24.941090 (+ 402us) server_negotiation.cc:407] Connection header received
0807 18:15:24.941267 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:24.941272 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:24.941343 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:24.941438 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:24.942460 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:24.943249 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:24.944738 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:24.945146 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:24.945300 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:24.955666 (+ 10366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.1:47845: BlockingRecv error: recv got EOF from 127.17.110.1:47845 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 07 18:15:25 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:25 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:25.432348 31025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:25.422627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.2:52115)
0807 18:15:25.423231 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:25.423248 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:25.423404 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:25.423784 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:25.423792 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:25.424273 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:25.424439 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:25.424448 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:25.425546 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:25.425552 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:25.426543 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:25.426557 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:25.427372 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:25.427379 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:25.427569 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:25.428257 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:25.428276 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:25.432111 (+ 3835us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":281,"threads_started":1}
W20260807 18:15:25.432698 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:25.422919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:52115 (local address 127.17.110.62:38611)
0807 18:15:25.423128 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:25.423134 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:25.423329 (+ 195us) server_negotiation.cc:407] Connection header received
0807 18:15:25.423555 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:25.423559 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:25.423629 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:25.423735 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:25.424603 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:25.425394 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:25.426694 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:25.427204 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:25.427355 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:25.432364 (+ 5009us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.2:52115: BlockingRecv error: recv got EOF from 127.17.110.2:52115 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 07 18:15:25 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:25 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:25.730999 31026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:25.720578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.3:43069)
0807 18:15:25.721161 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:25.721179 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:25.721303 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:25.721671 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:25.721675 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:25.722201 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:25.722387 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:25.722397 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:25.723501 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:25.723507 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:25.724624 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:25.724639 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:25.725381 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:25.725386 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:25.725539 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:25.726166 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:25.726192 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:25.730731 (+ 4539us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":263,"threads_started":1}
W20260807 18:15:25.731161 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:25.720848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:43069 (local address 127.17.110.62:38611)
0807 18:15:25.721080 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:25.721085 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:25.721273 (+ 188us) server_negotiation.cc:407] Connection header received
0807 18:15:25.721412 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:25.721415 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:25.721482 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:25.721568 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:25.722543 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:25.723344 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:25.724781 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:25.725209 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:25.725323 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:25.730954 (+ 5631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.3:43069: BlockingRecv error: recv got EOF from 127.17.110.3:43069 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 07 18:15:25 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:25 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:25.968168 31027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:25.956945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.1:51457)
0807 18:15:25.957597 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:25.957616 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:25.957816 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:25.958217 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:25.958222 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:25.958763 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:25.958928 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:25.958938 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:25.960106 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:25.960113 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:25.961457 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:25.961476 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:25.962490 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:25.962498 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:25.962816 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:25.963511 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:25.963531 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:25.967800 (+ 4269us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":367,"threads_started":1}
W20260807 18:15:25.968360 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:25.957089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:51457 (local address 127.17.110.62:38611)
0807 18:15:25.957442 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:25.957450 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:25.957732 (+ 282us) server_negotiation.cc:407] Connection header received
0807 18:15:25.957976 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:25.957981 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:25.958054 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:25.958181 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:25.959105 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:25.959937 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:25.961665 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:25.962228 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:25.962380 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:25.968088 (+ 5708us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.1:51457: BlockingRecv error: recv got EOF from 127.17.110.1:51457 (error 108)
Metrics: {"server-negotiator.queue_time_us":129}
Aug 07 18:15:26 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:26 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:26.445192 31029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:26.433652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.2:41223)
0807 18:15:26.434346 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:26.434376 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:26.434526 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:26.435301 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:26.435306 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:26.435822 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:26.436015 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:26.436027 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:26.437361 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:26.437366 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:26.438792 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:26.438810 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:26.439841 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:26.439848 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:26.440072 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:26.440866 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:26.440886 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:26.444854 (+ 3968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":345,"threads_started":1}
W20260807 18:15:26.445339 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:26.433813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:41223 (local address 127.17.110.62:38611)
0807 18:15:26.434014 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:26.434019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:26.434521 (+ 502us) server_negotiation.cc:407] Connection header received
0807 18:15:26.434953 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:26.434957 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:26.435035 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:26.435146 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:26.436264 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:26.437140 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:26.438969 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:26.439662 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:26.439770 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:26.445082 (+ 5312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.2:41223: BlockingRecv error: recv got EOF from 127.17.110.2:41223 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 07 18:15:26 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:26 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:26.742654 31033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:26.732442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.3:37439)
0807 18:15:26.733048 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:26.733064 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:26.733304 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:26.733688 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:26.733692 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:26.734164 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:26.734344 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:26.734353 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:26.735704 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:26.735709 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:26.737083 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:26.737104 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:26.737903 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:26.737908 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:26.738042 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:26.738688 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:26.738729 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:26.742406 (+ 3677us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":362,"threads_started":1}
W20260807 18:15:26.743070 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:26.732605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:37439 (local address 127.17.110.62:38611)
0807 18:15:26.732883 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:26.732890 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:26.733216 (+ 326us) server_negotiation.cc:407] Connection header received
0807 18:15:26.733456 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:26.733466 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:26.733534 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:26.733656 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:26.734508 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:26.735513 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:26.737266 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:26.737746 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:26.737848 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:26.742780 (+ 4932us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.3:37439: BlockingRecv error: recv got EOF from 127.17.110.3:37439 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 07 18:15:26 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:26 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:26.979601 31034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:26.969468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.1:39621)
0807 18:15:26.970227 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:26.970246 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:26.970373 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:26.970698 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:26.970702 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:26.971287 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:26.971494 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:26.971504 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:26.972603 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:26.972608 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:26.973606 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:26.973618 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:26.974456 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:26.974461 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:26.974602 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:26.975381 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:26.975401 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:26.979326 (+ 3925us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":423,"threads_started":1}
W20260807 18:15:26.979899 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:26.969559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:39621 (local address 127.17.110.62:38611)
0807 18:15:26.969834 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:26.969843 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:26.970335 (+ 492us) server_negotiation.cc:407] Connection header received
0807 18:15:26.970480 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:26.970483 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:26.970550 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:26.970639 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:26.971662 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:26.972435 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:26.973810 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:26.974266 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:26.974377 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:26.979606 (+ 5229us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.1:39621: BlockingRecv error: recv got EOF from 127.17.110.1:39621 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 07 18:15:27 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:27 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:27.458508 31036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:27.446754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.2:44257)
0807 18:15:27.447500 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:27.447517 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:27.447724 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:27.448116 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:27.448121 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:27.448612 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:27.448842 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:27.448852 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:27.450203 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:27.450210 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:27.451789 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:27.451807 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:27.452737 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:27.452742 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:27.452858 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:27.453630 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:27.453652 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:27.458200 (+ 4548us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":632,"thread_start_us":340,"threads_started":1}
W20260807 18:15:27.459080 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:27.446897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:44257 (local address 127.17.110.62:38611)
0807 18:15:27.447201 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:27.447210 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:27.447635 (+ 425us) server_negotiation.cc:407] Connection header received
0807 18:15:27.447887 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:27.447891 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:27.447963 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:27.448078 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:27.449042 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:27.450009 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:27.451978 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:27.452569 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:27.452701 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:27.458526 (+ 5825us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.2:44257: BlockingRecv error: recv got EOF from 127.17.110.2:44257 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Aug 07 18:15:27 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:27 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:27.755455 31038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:27.743955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.3:53591)
0807 18:15:27.744701 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:27.744719 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:27.744904 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:27.745303 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:27.745308 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:27.745964 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:27.746250 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:27.746262 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:27.747716 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:27.747721 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:27.748861 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:27.748876 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:27.749880 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:27.749887 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:27.750091 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:27.750939 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:27.750966 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:27.755106 (+ 4140us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":368,"threads_started":1}
W20260807 18:15:27.755802 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:27.744109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:53591 (local address 127.17.110.62:38611)
0807 18:15:27.744420 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:27.744428 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:27.744822 (+ 394us) server_negotiation.cc:407] Connection header received
0807 18:15:27.745062 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:27.745066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:27.745143 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:27.745257 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:27.746489 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:27.747470 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:27.749058 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:27.749618 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:27.749785 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:27.755476 (+ 5691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.3:53591: BlockingRecv error: recv got EOF from 127.17.110.3:53591 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 07 18:15:27 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:27 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:27.992274 31040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:27.980857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.1:46231)
0807 18:15:27.981643 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:27.981662 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:27.981847 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:27.982241 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:27.982246 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:27.982793 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:27.982991 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:27.983001 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:27.984171 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:27.984184 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:27.985169 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:27.985183 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:27.986030 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:27.986037 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:27.986231 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:27.987117 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:27.987142 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:27.991862 (+ 4720us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":504,"threads_started":1}
W20260807 18:15:27.992384 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:27.981133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:46231 (local address 127.17.110.62:38611)
0807 18:15:27.981432 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:27.981440 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:27.981767 (+ 327us) server_negotiation.cc:407] Connection header received
0807 18:15:27.981999 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:27.982004 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:27.982075 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:27.982194 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:27.983177 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:27.984009 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:27.985334 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:27.985790 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:27.985965 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:27.992124 (+ 6159us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.1:46231: BlockingRecv error: recv got EOF from 127.17.110.1:46231 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 07 18:15:28 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:28 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:28.470527 31041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:28.459707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.2:53383)
0807 18:15:28.460343 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:28.460368 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:28.460556 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:28.460950 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:28.460955 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:28.461490 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:28.461780 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:28.461795 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:28.463091 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:28.463097 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:28.464142 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:28.464156 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:28.464989 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:28.464996 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:28.465194 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:28.465857 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:28.465877 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:28.470099 (+ 4222us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":317,"threads_started":1}
W20260807 18:15:28.470800 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:28.459823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:53383 (local address 127.17.110.62:38611)
0807 18:15:28.460099 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:28.460106 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:28.460476 (+ 370us) server_negotiation.cc:407] Connection header received
0807 18:15:28.460718 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:28.460723 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:28.460800 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:28.460911 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:28.461940 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:28.462927 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:28.464298 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:28.464789 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:28.464942 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:28.470444 (+ 5502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.2:53383: BlockingRecv error: recv got EOF from 127.17.110.2:53383 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 07 18:15:28 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:28 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:28.766913 31042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:28.756596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.3:55179)
0807 18:15:28.757159 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:28.757178 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:28.757319 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:28.757737 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:28.757742 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:28.758223 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:28.758469 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:28.758481 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:28.760089 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:28.760094 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:28.761096 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:28.761112 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:28.762077 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:28.762082 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:28.762203 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:28.762934 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:28.762954 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:28.766566 (+ 3612us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":289,"threads_started":1}
W20260807 18:15:28.767268 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:28.756672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:55179 (local address 127.17.110.62:38611)
0807 18:15:28.756947 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:28.756954 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:28.757278 (+ 324us) server_negotiation.cc:407] Connection header received
0807 18:15:28.757455 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:28.757461 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:28.757553 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:28.757698 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:28.758682 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:28.759922 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:28.761271 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:28.761896 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:28.762019 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:28.766920 (+ 4901us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.3:55179: BlockingRecv error: recv got EOF from 127.17.110.3:55179 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 07 18:15:29 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:29 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:29.007926 31043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:28.993485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.1:40657)
0807 18:15:28.994085 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:28.994102 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:28.994274 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:28.994634 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:28.994639 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:28.995188 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:28.995350 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:28.995360 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:28.996428 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:28.996441 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:28.997408 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:28.997444 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:28.998205 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:28.998211 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:28.998400 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:29.003722 (+ 5322us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:29.003746 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:29.007661 (+ 3915us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":327,"threads_started":1}
W20260807 18:15:29.008165 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:28.993587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:40657 (local address 127.17.110.62:38611)
0807 18:15:28.993897 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:28.993907 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:28.994199 (+ 292us) server_negotiation.cc:407] Connection header received
0807 18:15:28.994412 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:28.994419 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:28.994490 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:28.994610 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:28.995513 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:28.996274 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:28.997591 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:28.998021 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:28.998163 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:29.007915 (+ 9752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.1:40657: BlockingRecv error: recv got EOF from 127.17.110.1:40657 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
Aug 07 18:15:29 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:29 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:29.483214 31044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:29.471932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.2:37635)
0807 18:15:29.472608 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:29.472625 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:29.472803 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:29.473196 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:29.473200 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:29.473681 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:29.473836 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:29.473846 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:29.475100 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:29.475113 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:29.476110 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:29.476125 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:29.476962 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:29.476968 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:29.477210 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:29.478387 (+ 1177us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:29.478406 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:29.482944 (+ 4538us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":368,"threads_started":1}
W20260807 18:15:29.483616 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:29.472088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:37635 (local address 127.17.110.62:38611)
0807 18:15:29.472381 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:29.472388 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:29.472730 (+ 342us) server_negotiation.cc:407] Connection header received
0807 18:15:29.472958 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:29.472964 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:29.473031 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:29.473154 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:29.474022 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:29.474935 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:29.476272 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:29.476753 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:29.476909 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:29.483252 (+ 6343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.2:37635: BlockingRecv error: recv got EOF from 127.17.110.2:37635 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Aug 07 18:15:29 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:29 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:29.780012 31045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:29.768210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.3:60301)
0807 18:15:29.768761 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:29.768779 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:29.768914 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:29.769302 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:29.769306 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:29.769799 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:29.769983 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:29.769993 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:29.771263 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:29.771270 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:29.772744 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:29.772762 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:29.773767 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:29.773775 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:29.773982 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:29.774688 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:29.774729 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:29.779758 (+ 5029us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":302,"threads_started":1}
W20260807 18:15:29.780316 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:29.768297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:60301 (local address 127.17.110.62:38611)
0807 18:15:29.768600 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:29.768606 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:29.768877 (+ 271us) server_negotiation.cc:407] Connection header received
0807 18:15:29.769038 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:29.769042 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:29.769114 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:29.769223 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:29.770154 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:29.771026 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:29.772919 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:29.773580 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:29.773741 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:29.780059 (+ 6318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.3:60301: BlockingRecv error: recv got EOF from 127.17.110.3:60301 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 07 18:15:30 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:30 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:30.027581 31047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:30.009291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.1:35625)
0807 18:15:30.009863 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:30.009881 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:30.010010 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:30.010399 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:30.010403 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:30.010913 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:30.011132 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:30.011143 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:30.012824 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:30.012833 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:30.014891 (+ 2058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:30.014922 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:30.016247 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:30.016255 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:30.016464 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:30.022604 (+ 6140us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:30.022632 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:30.027189 (+ 4557us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":300,"threads_started":1}
W20260807 18:15:30.027905 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:30.009406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:35625 (local address 127.17.110.62:38611)
0807 18:15:30.009695 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:30.009701 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:30.010016 (+ 315us) server_negotiation.cc:407] Connection header received
0807 18:15:30.010126 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:30.010130 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:30.010223 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:30.010338 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:30.011306 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:30.012625 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:30.015093 (+ 2468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:30.015992 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:30.016159 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:30.027481 (+ 11322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.1:35625: BlockingRecv error: recv got EOF from 127.17.110.1:35625 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 07 18:15:30 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:30 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:30.494910 31048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:30.484419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.2:43519)
0807 18:15:30.485087 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:30.485105 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:30.485288 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:30.485711 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:30.485716 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:30.486160 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:30.486322 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:30.486332 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:30.487640 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:30.487649 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:30.488943 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:30.488960 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:30.489751 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:30.489756 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:30.489894 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:30.490682 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:30.490725 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:30.494568 (+ 3843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":341,"threads_started":1}
W20260807 18:15:30.495244 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:30.484666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:43519 (local address 127.17.110.62:38611)
0807 18:15:30.484949 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:30.484957 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:30.485207 (+ 250us) server_negotiation.cc:407] Connection header received
0807 18:15:30.485447 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:30.485454 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:30.485533 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:30.485667 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:30.486500 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:30.487460 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:30.489140 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:30.489577 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:30.489696 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:30.494923 (+ 5227us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.2:43519: BlockingRecv error: recv got EOF from 127.17.110.2:43519 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Aug 07 18:15:30 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:30 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:30.791338 31049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:30.781464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.3:43911)
0807 18:15:30.782050 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:30.782072 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:30.782258 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:30.782648 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:30.782653 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:30.783161 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:30.783342 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:30.783352 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:30.784468 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:30.784474 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:30.785505 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:30.785519 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:30.786353 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:30.786359 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:30.786544 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:30.787247 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:30.787267 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:30.791067 (+ 3800us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":403,"threads_started":1}
W20260807 18:15:30.791678 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:30.781608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:43911 (local address 127.17.110.62:38611)
0807 18:15:30.781864 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:30.781871 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:30.782179 (+ 308us) server_negotiation.cc:407] Connection header received
0807 18:15:30.782409 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:30.782415 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:30.782492 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:30.782605 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:30.783516 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:30.784295 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:30.785679 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:30.786151 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:30.786324 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:30.791376 (+ 5052us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.3:43911: BlockingRecv error: recv got EOF from 127.17.110.3:43911 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 07 18:15:31 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:31 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:31.039510 31053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:31.029244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.1:60113)
0807 18:15:31.029778 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:31.029796 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:31.029929 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:31.030350 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:31.030354 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:31.030875 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:31.031085 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:31.031094 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:31.032142 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:31.032146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:31.033096 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:31.033109 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:31.033844 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:31.033848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:31.033978 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:31.034624 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:31.034645 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:31.039235 (+ 4590us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":271,"threads_started":1}
W20260807 18:15:31.039636 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:31.029461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:60113 (local address 127.17.110.62:38611)
0807 18:15:31.029741 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:31.029753 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:31.029886 (+ 133us) server_negotiation.cc:407] Connection header received
0807 18:15:31.030087 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:31.030092 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:31.030165 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:31.030261 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:31.031231 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:31.031993 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:31.033245 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:31.033688 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:31.033797 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:31.039437 (+ 5640us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.1:60113: BlockingRecv error: recv got EOF from 127.17.110.1:60113 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
Aug 07 18:15:31 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:31 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:31.505167 31054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:31.495998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.2:45783)
0807 18:15:31.496460 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:31.496477 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:31.496594 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:31.496915 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:31.496919 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:31.497382 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:31.497563 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:31.497572 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:31.498632 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:31.498636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:31.499606 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:31.499619 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:31.500334 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:31.500340 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:31.500465 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:31.501204 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:31.501222 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:31.504928 (+ 3706us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":248,"threads_started":1}
W20260807 18:15:31.505573 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:31.496093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:45783 (local address 127.17.110.62:38611)
0807 18:15:31.496378 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:31.496384 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:31.496555 (+ 171us) server_negotiation.cc:407] Connection header received
0807 18:15:31.496706 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:31.496709 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:31.496784 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:31.496867 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:31.497714 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:31.498509 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:31.499747 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:31.500163 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:31.500279 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:31.505195 (+ 4916us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.2:45783: BlockingRecv error: recv got EOF from 127.17.110.2:45783 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Aug 07 18:15:31 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:31 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:31.802366 31056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:31.792522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.3:48075)
0807 18:15:31.793133 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:31.793153 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:31.793289 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:31.793681 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:31.793687 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:31.794233 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:31.794469 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:31.794480 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:31.795570 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:31.795575 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:31.796554 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:31.796568 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:31.797341 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:31.797345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:31.797473 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:31.798160 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:31.798180 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:31.802124 (+ 3944us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":354,"threads_started":1}
W20260807 18:15:31.802657 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:31.792617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:48075 (local address 127.17.110.62:38611)
0807 18:15:31.792826 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:31.792831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:31.793274 (+ 443us) server_negotiation.cc:407] Connection header received
0807 18:15:31.793414 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:31.793418 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:31.793488 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:31.793581 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:31.794629 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:31.795409 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:31.796715 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:31.797187 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:31.797298 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:31.802429 (+ 5131us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.3:48075: BlockingRecv error: recv got EOF from 127.17.110.3:48075 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 07 18:15:32 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:32 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:32.050524 31058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:32.040663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.1:56195)
0807 18:15:32.041254 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:32.041272 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:32.041391 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:32.041833 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:32.041837 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:32.042248 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:32.042433 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:32.042442 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:32.043583 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:32.043588 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:32.044518 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:32.044539 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:32.045527 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:32.045532 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:32.045716 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:32.046502 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:32.046530 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:32.050264 (+ 3734us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":322,"threads_started":1}
W20260807 18:15:32.050856 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:32.040773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:56195 (local address 127.17.110.62:38611)
0807 18:15:32.041048 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:32.041055 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:32.041360 (+ 305us) server_negotiation.cc:407] Connection header received
0807 18:15:32.041567 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:32.041571 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:32.041646 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:32.041749 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:32.042586 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:32.043418 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:32.044691 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:32.045371 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:32.045514 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:32.050504 (+ 4990us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.1:56195: BlockingRecv error: recv got EOF from 127.17.110.1:56195 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Aug 07 18:15:32 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:32 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:32.516146 31059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:32.506428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.2:46003)
0807 18:15:32.507058 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:32.507075 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:32.507265 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:32.507665 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:32.507670 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:32.508123 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:32.508298 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:32.508308 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:32.509397 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:32.509403 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:32.510358 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:32.510371 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:32.511187 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:32.511193 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:32.511395 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:32.512032 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:32.512051 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:32.515759 (+ 3708us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":351,"threads_started":1}
W20260807 18:15:32.516538 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:32.506590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:46003 (local address 127.17.110.62:38611)
0807 18:15:32.506867 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:32.506874 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:32.507194 (+ 320us) server_negotiation.cc:407] Connection header received
0807 18:15:32.507420 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:32.507427 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:32.507505 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:32.507620 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:32.508486 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:32.509247 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:32.510512 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:32.511003 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:32.511168 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:32.516089 (+ 4921us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.2:46003: BlockingRecv error: recv got EOF from 127.17.110.2:46003 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 07 18:15:32 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:32 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:32.813900 31060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:32.803649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.3:38417)
0807 18:15:32.804313 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:32.804330 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:32.804501 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:32.804872 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:32.804877 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:32.805353 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:32.805530 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:32.805540 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:32.806625 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:32.806631 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:32.807613 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:32.807625 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:32.808510 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:32.808516 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:32.808704 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:32.809400 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:32.809419 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:32.813618 (+ 4199us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":344,"threads_started":1}
W20260807 18:15:32.814158 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:32.803783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:38417 (local address 127.17.110.62:38611)
0807 18:15:32.804044 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:32.804052 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:32.804429 (+ 377us) server_negotiation.cc:407] Connection header received
0807 18:15:32.804643 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:32.804654 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:32.804726 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:32.804836 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:32.805719 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:32.806476 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:32.807759 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:32.808316 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:32.808459 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:32.813872 (+ 5413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.3:38417: BlockingRecv error: recv got EOF from 127.17.110.3:38417 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 07 18:15:33 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:33 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:33.061622 31062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:33.051829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.1:43779)
0807 18:15:33.052406 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:33.052427 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:33.052616 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:33.052983 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:33.052989 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:33.053548 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:33.053703 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:33.053713 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:33.054870 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:33.054877 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:33.055964 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:33.055978 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:33.056722 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:33.056728 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:33.056954 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:33.057589 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:33.057608 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:33.061321 (+ 3713us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":410,"threads_started":1}
W20260807 18:15:33.061846 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:33.051922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:43779 (local address 127.17.110.62:38611)
0807 18:15:33.052236 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:33.052244 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:33.052503 (+ 259us) server_negotiation.cc:407] Connection header received
0807 18:15:33.052755 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:33.052761 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:33.052838 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:33.052950 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:33.053858 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:33.054689 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:33.056124 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:33.056565 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:33.056708 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:33.061574 (+ 4866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.1:43779: BlockingRecv error: recv got EOF from 127.17.110.1:43779 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 07 18:15:33 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:33 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.2@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:33.528781 31063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:33.517612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.2:52897)
0807 18:15:33.518780 (+ 1168us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:33.518804 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:33.518941 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:33.519357 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:33.519360 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:33.519786 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:33.519947 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:33.519957 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:33.521122 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:33.521127 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:33.522170 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:33.522186 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:33.523038 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:33.523043 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:33.523198 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:33.524004 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:33.524033 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:33.528523 (+ 4490us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1011,"thread_start_us":850,"threads_started":1}
W20260807 18:15:33.529131 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:33.517879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:52897 (local address 127.17.110.62:38611)
0807 18:15:33.518108 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:33.518113 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:33.518907 (+ 794us) server_negotiation.cc:407] Connection header received
0807 18:15:33.519066 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:33.519070 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:33.519140 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:33.519239 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:33.520109 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:33.520945 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:33.522366 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:33.522874 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:33.522985 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:33.528856 (+ 5871us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.2:52897: BlockingRecv error: recv got EOF from 127.17.110.2:52897 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 07 18:15:33 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:33 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.3@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:33.827303 31064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:33.815110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.3:34253)
0807 18:15:33.815879 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:33.815897 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:33.816019 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:33.816555 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:33.816559 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:33.817006 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:33.817174 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:33.817185 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:33.818512 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:33.818518 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:33.819735 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:33.819754 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:33.820671 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:33.820677 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:33.820855 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:33.821806 (+ 951us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:33.821828 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:33.826961 (+ 5133us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":525,"threads_started":1}
W20260807 18:15:33.827502 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:33.815262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:34253 (local address 127.17.110.62:38611)
0807 18:15:33.815473 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:33.815479 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:33.815999 (+ 520us) server_negotiation.cc:407] Connection header received
0807 18:15:33.816256 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:33.816261 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:33.816332 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:33.816432 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:33.817360 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:33.818254 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:33.819971 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:33.820452 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:33.820574 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:33.827228 (+ 6654us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.3:34253: BlockingRecv error: recv got EOF from 127.17.110.3:34253 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 07 18:15:34 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:15:34 dist-test-slave-39l8 krb5kdc[29494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.110.1@KRBTEST.COM for oryx/127.17.110.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:15:34.074870 31066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:34.063297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38611 (local address 127.17.110.1:41443)
0807 18:15:34.063905 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:34.063930 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:34.064116 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:34.064678 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:34.064682 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:34.065119 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:34.065494 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:34.065507 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:34.067051 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:34.067059 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:34.068273 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:34.068293 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:34.069365 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:34.069371 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:34.069520 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:34.070283 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:34.070305 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:34.074508 (+ 4203us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.110.62:38611: Server oryx/127.17.110.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":330,"threads_started":1}
W20260807 18:15:34.075213 30999 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:34.063509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:41443 (local address 127.17.110.62:38611)
0807 18:15:34.063857 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:34.063862 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:34.064071 (+ 209us) server_negotiation.cc:407] Connection header received
0807 18:15:34.064322 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:34.064327 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:34.064428 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:34.064661 (+ 233us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:34.065810 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:34.066834 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:34.068468 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:34.069154 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:34.069277 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:34.074872 (+ 5595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.110.1:41443: BlockingRecv error: recv got EOF from 127.17.110.1:41443 (error 108)
Metrics: {"server-negotiator.queue_time_us":145}
I20260807 18:15:34.431110 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 30369
I20260807 18:15:34.447549 31069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:34.432776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36706 (local address 127.17.110.1:38351)
0807 18:15:34.433661 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:34.433669 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:34.433707 (+ 38us) server_negotiation.cc:407] Connection header received
0807 18:15:34.433791 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:34.433798 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:34.434082 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:34.434293 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:34.436265 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:34.437851 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:34.439570 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:34.440197 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:34.440368 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:34.443789 (+ 3421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:34.443800 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:34.443818 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:34.443880 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:34.446929 (+ 3049us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.17.110.1@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 18:15:34.447292 (+ 363us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36706: Request ticket server kudu/127.17.110.1@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":460,"threads_started":1}
W20260807 18:15:34.447647 31069 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36706: Request ticket server kudu/127.17.110.1@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 18:15:34.447939 17848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 30369: Not authorized: Client connection negotiation failed: client connection to 127.17.110.1:38351: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.17.110.1@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 18:15:34.448138 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 30369
I20260807 18:15:34.474965 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 30607
I20260807 18:15:34.488591 31070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:34.476281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52458 (local address 127.17.110.2:39667)
0807 18:15:34.477045 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:34.477055 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:34.477089 (+ 34us) server_negotiation.cc:407] Connection header received
0807 18:15:34.477168 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:34.477180 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:34.477507 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:34.477757 (+ 250us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:34.479338 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:34.480378 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:34.481635 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:34.482202 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:34.482324 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:34.485028 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:34.485037 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:34.485054 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:34.485102 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:34.487975 (+ 2873us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.17.110.2@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 18:15:34.488300 (+ 325us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52458: Request ticket server kudu/127.17.110.2@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":367,"threads_started":1}
W20260807 18:15:34.488718 31070 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52458: Request ticket server kudu/127.17.110.2@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 18:15:34.488808 17848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 30607: Not authorized: Client connection negotiation failed: client connection to 127.17.110.2:39667: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.17.110.2@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 18:15:34.488991 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 30607
I20260807 18:15:34.516726 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 30795
I20260807 18:15:34.530982 31071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:34.517938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33426 (local address 127.17.110.3:33189)
0807 18:15:34.518673 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:34.518683 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:34.518752 (+ 69us) server_negotiation.cc:407] Connection header received
0807 18:15:34.518839 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:34.518848 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:34.519187 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:34.519423 (+ 236us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:34.520830 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:34.521945 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:34.523392 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:34.524000 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:34.524111 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:34.526953 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:34.526964 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:34.526981 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:34.527033 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:34.530064 (+ 3031us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.17.110.3@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 18:15:34.530511 (+ 447us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33426: Request ticket server kudu/127.17.110.3@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":367,"threads_started":1}
W20260807 18:15:34.531139 31071 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33426: Request ticket server kudu/127.17.110.3@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 18:15:34.531275 17848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 30795: Not authorized: Client connection negotiation failed: client connection to 127.17.110.3:33189: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.17.110.3@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 18:15:34.531507 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 30795
I20260807 18:15:34.554774 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 30267
I20260807 18:15:34.556391 30361 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:34.678076 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 30267
2026-08-07T18:15:34Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (22324 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T18:15:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:15:34Z Disabled control of system clock
I20260807 18:15:34.780860 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:46645
--webserver_interface=127.17.110.62
--webserver_port=0
--builtin_ntp_servers=127.17.110.20:38665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:46645
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 18:15:35.145012 31079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:35.145367 31079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:35.145516 31079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:35.154119 31079 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:15:35.154300 31079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:35.154409 31079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:35.154518 31079 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:15:35.154577 31079 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:15:35.165004 31079 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:38665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:46645
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:46645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:15:35.167124 31079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:35.169293 31079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:35.182390 31086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:35.182430 31088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:35.182390 31085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:35.183538 31079 server_base.cc:1034] running on GCE node
I20260807 18:15:35.184419 31079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:35.186002 31079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:35.187256 31079 hybrid_clock.cc:648] HybridClock initialized: now 1786126535187211 us; error 62 us; skew 500 ppm
I20260807 18:15:35.187647 31079 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:15:35.194325 31079 webserver.cc:460] Webserver started at http://127.17.110.62:34719/ using document root <none> and password file <none>
I20260807 18:15:35.195230 31079 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:35.195384 31079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:35.195752 31079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:15:35.199038 31079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "e942b5363cc54a2ab2053f50affed855"
format_stamp: "Formatted at 2026-08-07 18:15:35 on dist-test-slave-39l8"
server_key: "cb3ad7c565ac63cb2923a781c945c36c"
server_key_iv: "ad73e9972d6f6d5711dc9e5ed3612298"
server_key_version: "encryptionkey@0"
I20260807 18:15:35.199909 31079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "e942b5363cc54a2ab2053f50affed855"
format_stamp: "Formatted at 2026-08-07 18:15:35 on dist-test-slave-39l8"
server_key: "cb3ad7c565ac63cb2923a781c945c36c"
server_key_iv: "ad73e9972d6f6d5711dc9e5ed3612298"
server_key_version: "encryptionkey@0"
I20260807 18:15:35.206440 31079 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 18:15:35.210625 31096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:35.212328 31079 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:15:35.212567 31079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "e942b5363cc54a2ab2053f50affed855"
format_stamp: "Formatted at 2026-08-07 18:15:35 on dist-test-slave-39l8"
server_key: "cb3ad7c565ac63cb2923a781c945c36c"
server_key_iv: "ad73e9972d6f6d5711dc9e5ed3612298"
server_key_version: "encryptionkey@0"
I20260807 18:15:35.212837 31079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:35.234084 31079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:35.235198 31079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:35.235546 31079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:35.256747 31079 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:46645
I20260807 18:15:35.256786 31165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:46645 every 8 connection(s)
I20260807 18:15:35.258291 31079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:15:35.261873 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 31079
I20260807 18:15:35.262132 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:15:35.262904 31166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:35.262946 17848 external_mini_cluster.cc:1428] Setting key e110fdef4f8649e103098dabe36fe946
I20260807 18:15:35.281152 31166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e942b5363cc54a2ab2053f50affed855: Bootstrap starting.
I20260807 18:15:35.281567 31172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:35.265316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55186 (local address 127.17.110.62:46645)
0807 18:15:35.266631 (+ 1315us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:35.266656 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:35.266750 (+ 94us) server_negotiation.cc:407] Connection header received
0807 18:15:35.268944 (+ 2194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:35.268983 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:35.269788 (+ 805us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:35.270365 (+ 577us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:35.271137 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:35.273459 (+ 2322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:35.275212 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:35.276443 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:35.276708 (+ 265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:35.278025 (+ 1317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:35.278037 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:35.278070 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:15:35.278969 (+ 899us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:35.279102 (+ 133us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:15:35.279133 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:35.279280 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:35.279584 (+ 304us) server_negotiation.cc:299] Negotiation successful
0807 18:15:35.279980 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":372,"threads_started":1}
I20260807 18:15:35.285145 31166 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e942b5363cc54a2ab2053f50affed855: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:35.286293 31166 log.cc:824] T 00000000000000000000000000000000 P e942b5363cc54a2ab2053f50affed855: Log is configured to *not* fsync() on all Append() calls
I20260807 18:15:35.290015 31166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e942b5363cc54a2ab2053f50affed855: No bootstrap required, opened a new log
I20260807 18:15:35.298915 31166 raft_consensus.cc:348] T 00000000000000000000000000000000 P e942b5363cc54a2ab2053f50affed855 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e942b5363cc54a2ab2053f50affed855" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 46645 } }
I20260807 18:15:35.299355 31166 raft_consensus.cc:374] T 00000000000000000000000000000000 P e942b5363cc54a2ab2053f50affed855 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:35.299440 31166 raft_consensus.cc:729] T 00000000000000000000000000000000 P e942b5363cc54a2ab2053f50affed855 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e942b5363cc54a2ab2053f50affed855, State: Initialized, Role: FOLLOWER
I20260807 18:15:35.300107 31166 consensus_queue.cc:260] T 00000000000000000000000000000000 P e942b5363cc54a2ab2053f50affed855 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e942b5363cc54a2ab2053f50affed855" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 46645 } }
I20260807 18:15:35.300383 31166 raft_consensus.cc:388] T 00000000000000000000000000000000 P e942b5363cc54a2ab2053f50affed855 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:15:35.300506 31166 raft_consensus.cc:482] T 00000000000000000000000000000000 P e942b5363cc54a2ab2053f50affed855 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:15:35.300660 31166 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e942b5363cc54a2ab2053f50affed855 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:35.303642 31166 raft_consensus.cc:504] T 00000000000000000000000000000000 P e942b5363cc54a2ab2053f50affed855 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e942b5363cc54a2ab2053f50affed855" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 46645 } }
I20260807 18:15:35.304181 31166 leader_election.cc:302] T 00000000000000000000000000000000 P e942b5363cc54a2ab2053f50affed855 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e942b5363cc54a2ab2053f50affed855; no voters:
I20260807 18:15:35.304816 31166 leader_election.cc:288] T 00000000000000000000000000000000 P e942b5363cc54a2ab2053f50affed855 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:15:35.305028 31175 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e942b5363cc54a2ab2053f50affed855 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:15:35.305887 31175 raft_consensus.cc:686] T 00000000000000000000000000000000 P e942b5363cc54a2ab2053f50affed855 [term 1 LEADER]: Becoming Leader. State: Replica: e942b5363cc54a2ab2053f50affed855, State: Running, Role: LEADER
I20260807 18:15:35.306479 31175 consensus_queue.cc:237] T 00000000000000000000000000000000 P e942b5363cc54a2ab2053f50affed855 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e942b5363cc54a2ab2053f50affed855" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 46645 } }
I20260807 18:15:35.307324 31166 sys_catalog.cc:564] T 00000000000000000000000000000000 P e942b5363cc54a2ab2053f50affed855 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:15:35.310118 31177 sys_catalog.cc:455] T 00000000000000000000000000000000 P e942b5363cc54a2ab2053f50affed855 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e942b5363cc54a2ab2053f50affed855. Latest consensus state: current_term: 1 leader_uuid: "e942b5363cc54a2ab2053f50affed855" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e942b5363cc54a2ab2053f50affed855" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 46645 } } }
I20260807 18:15:35.310672 31177 sys_catalog.cc:458] T 00000000000000000000000000000000 P e942b5363cc54a2ab2053f50affed855 [sys.catalog]: This master's current role is: LEADER
I20260807 18:15:35.310724 31176 sys_catalog.cc:455] T 00000000000000000000000000000000 P e942b5363cc54a2ab2053f50affed855 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e942b5363cc54a2ab2053f50affed855" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e942b5363cc54a2ab2053f50affed855" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 46645 } } }
I20260807 18:15:35.311405 31176 sys_catalog.cc:458] T 00000000000000000000000000000000 P e942b5363cc54a2ab2053f50affed855 [sys.catalog]: This master's current role is: LEADER
I20260807 18:15:35.313340 31186 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:15:35.318782 31186 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:15:35.329495 31186 catalog_manager.cc:1347] Generated new cluster ID: 1b0530d373f345a99c6320333e96466d
I20260807 18:15:35.329632 31186 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:15:35.340693 31186 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:15:35.342077 31186 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:15:35.352025 31186 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e942b5363cc54a2ab2053f50affed855: Generated new TSK 0
I20260807 18:15:35.353315 31186 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:15:35.368583 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.1:0
--local_ip_for_outbound_sockets=127.17.110.1
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:46645
--builtin_ntp_servers=127.17.110.20:38665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 18:15:35.744720 31200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:35.745064 31200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:35.745225 31200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:35.752790 31200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:35.752912 31200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:35.753101 31200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.1
I20260807 18:15:35.762647 31200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:38665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:46645
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.1
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:15:35.764199 31200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:35.766067 31200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:35.781780 31208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:35.781834 31212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:35.781834 31210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:35.783334 31200 server_base.cc:1034] running on GCE node
I20260807 18:15:35.784049 31200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:35.785136 31200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:35.786394 31200 hybrid_clock.cc:648] HybridClock initialized: now 1786126535786335 us; error 51 us; skew 500 ppm
I20260807 18:15:35.786828 31200 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:15:35.789567 31200 webserver.cc:460] Webserver started at http://127.17.110.1:37775/ using document root <none> and password file <none>
I20260807 18:15:35.790380 31200 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:35.790510 31200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:35.790959 31200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:15:35.794200 31200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/data/instance:
uuid: "c86b543476f7479eb10cd60f80a0e7ec"
format_stamp: "Formatted at 2026-08-07 18:15:35 on dist-test-slave-39l8"
server_key: "4af9bc36ee0f781b2701e9b1b648d39e"
server_key_iv: "c0025cbbaeb683898263d33272d05750"
server_key_version: "encryptionkey@0"
I20260807 18:15:35.795209 31200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance:
uuid: "c86b543476f7479eb10cd60f80a0e7ec"
format_stamp: "Formatted at 2026-08-07 18:15:35 on dist-test-slave-39l8"
server_key: "4af9bc36ee0f781b2701e9b1b648d39e"
server_key_iv: "c0025cbbaeb683898263d33272d05750"
server_key_version: "encryptionkey@0"
I20260807 18:15:35.801335 31200 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.002s
I20260807 18:15:35.805372 31223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:35.807035 31200 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:15:35.807281 31200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/wal
uuid: "c86b543476f7479eb10cd60f80a0e7ec"
format_stamp: "Formatted at 2026-08-07 18:15:35 on dist-test-slave-39l8"
server_key: "4af9bc36ee0f781b2701e9b1b648d39e"
server_key_iv: "c0025cbbaeb683898263d33272d05750"
server_key_version: "encryptionkey@0"
I20260807 18:15:35.807582 31200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:35.832798 31200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:35.833748 31200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:35.834096 31200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:35.835321 31200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:15:35.836892 31200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:15:35.837029 31200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:35.837160 31200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:15:35.837256 31200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:35.873556 31200 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.1:45891
I20260807 18:15:35.873661 31400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.1:45891 every 8 connection(s)
I20260807 18:15:35.875043 31200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
I20260807 18:15:35.881270 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 31200
I20260807 18:15:35.881484 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance
I20260807 18:15:35.882160 17848 external_mini_cluster.cc:1428] Setting key 60d3961cc42552310d2bc39b9c62f9b4
I20260807 18:15:35.885058 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.2:0
--local_ip_for_outbound_sockets=127.17.110.2
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:46645
--builtin_ntp_servers=127.17.110.20:38665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 18:15:35.888753 31404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:35.877938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:60881 (local address 127.17.110.62:46645)
0807 18:15:35.878481 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:35.878488 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:35.879731 (+ 1243us) server_negotiation.cc:407] Connection header received
0807 18:15:35.881608 (+ 1877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:35.881614 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:35.881722 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:35.881850 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:35.883200 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:35.883981 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:35.885689 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:35.886104 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:35.886266 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:35.887798 (+ 1532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:35.887803 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:35.887806 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:15:35.887827 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:35.887875 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:15:35.887904 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:35.888024 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:35.888285 (+ 261us) server_negotiation.cc:299] Negotiation successful
0807 18:15:35.888389 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":255,"threads_started":1}
I20260807 18:15:35.889839 31406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:35.878506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:46645 (local address 127.17.110.1:60881)
0807 18:15:35.879545 (+ 1039us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:35.879583 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:35.881270 (+ 1687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:35.882105 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:35.882113 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:35.882215 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:35.883002 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:35.883030 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:35.884168 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:35.884173 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:35.885528 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:35.885541 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:35.886361 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:35.886368 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:35.886612 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:35.887425 (+ 813us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:15:35.887428 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:35.887481 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:15:35.887573 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:35.888025 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:35.888039 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:35.888050 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 18:15:35.888243 (+ 193us) client_negotiation.cc:240] Negotiation successful
0807 18:15:35.888681 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":351,"threads_started":1}
I20260807 18:15:35.891953 31401 heartbeater.cc:347] Connected to a master server at 127.17.110.62:46645
I20260807 18:15:35.892387 31401 heartbeater.cc:464] Registering TS with master...
I20260807 18:15:35.893239 31401 heartbeater.cc:511] Master 127.17.110.62:46645 requested a full tablet report, sending...
I20260807 18:15:35.895781 31117 ts_manager.cc:194] Registered new tserver with Master: c86b543476f7479eb10cd60f80a0e7ec (127.17.110.1:45891)
I20260807 18:15:35.897406 31117 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.110.1:60881
W20260807 18:15:36.253108 31407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:36.253508 31407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:36.253679 31407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:36.261636 31407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:36.261833 31407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:36.262030 31407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.2
I20260807 18:15:36.271996 31407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:38665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:46645
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.2
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:15:36.273460 31407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:36.275470 31407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:36.290201 31413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:36.290261 31417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:36.290153 31414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:36.291638 31407 server_base.cc:1034] running on GCE node
I20260807 18:15:36.292320 31407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:36.293326 31407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:36.294554 31407 hybrid_clock.cc:648] HybridClock initialized: now 1786126536294483 us; error 57 us; skew 500 ppm
I20260807 18:15:36.294973 31407 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:15:36.297641 31407 webserver.cc:460] Webserver started at http://127.17.110.2:33181/ using document root <none> and password file <none>
I20260807 18:15:36.298487 31407 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:36.298611 31407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:36.299026 31407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:15:36.302167 31407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/data/instance:
uuid: "d600c800abc74751b2cfd20d5717cb1b"
format_stamp: "Formatted at 2026-08-07 18:15:36 on dist-test-slave-39l8"
server_key: "456cb7d401367b84e257005b9267ec15"
server_key_iv: "d21baa398993fcd71377a1e55dd29de2"
server_key_version: "encryptionkey@0"
I20260807 18:15:36.303141 31407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance:
uuid: "d600c800abc74751b2cfd20d5717cb1b"
format_stamp: "Formatted at 2026-08-07 18:15:36 on dist-test-slave-39l8"
server_key: "456cb7d401367b84e257005b9267ec15"
server_key_iv: "d21baa398993fcd71377a1e55dd29de2"
server_key_version: "encryptionkey@0"
I20260807 18:15:36.309639 31407 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 18:15:36.313953 31424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:36.315676 31407 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:15:36.315893 31407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/wal
uuid: "d600c800abc74751b2cfd20d5717cb1b"
format_stamp: "Formatted at 2026-08-07 18:15:36 on dist-test-slave-39l8"
server_key: "456cb7d401367b84e257005b9267ec15"
server_key_iv: "d21baa398993fcd71377a1e55dd29de2"
server_key_version: "encryptionkey@0"
I20260807 18:15:36.316152 31407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:36.359663 31407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:36.360642 31407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:36.360978 31407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:36.362071 31407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:15:36.363794 31407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:15:36.363889 31407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:36.364050 31407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:15:36.364141 31407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:36.403653 31407 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.2:46823
I20260807 18:15:36.403748 31591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.2:46823 every 8 connection(s)
I20260807 18:15:36.405161 31407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
I20260807 18:15:36.410516 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 31407
I20260807 18:15:36.410774 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance
I20260807 18:15:36.411432 17848 external_mini_cluster.cc:1428] Setting key 6f469dfe2b1c51aec87d2a71b84dc63f
I20260807 18:15:36.415517 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.3:0
--local_ip_for_outbound_sockets=127.17.110.3
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:46645
--builtin_ntp_servers=127.17.110.20:38665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 18:15:36.422863 31595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:36.408321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:53637 (local address 127.17.110.62:46645)
0807 18:15:36.408814 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:36.408818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:36.410318 (+ 1500us) server_negotiation.cc:407] Connection header received
0807 18:15:36.413153 (+ 2835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:36.413159 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:36.413279 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:36.413413 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:36.415202 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:36.416181 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:36.419150 (+ 2969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:36.419634 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:36.419767 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:36.421751 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:36.421757 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:36.421761 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:15:36.421785 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:36.421832 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:15:36.421847 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:36.421980 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:36.422311 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 18:15:36.422433 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":266,"threads_started":1}
I20260807 18:15:36.424011 31598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:36.408921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:46645 (local address 127.17.110.2:53637)
0807 18:15:36.410035 (+ 1114us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:36.410090 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:36.412749 (+ 2659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:36.413662 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:36.413671 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:36.413940 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:36.414936 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:36.414968 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:36.416381 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:36.416390 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:36.418940 (+ 2550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:36.418967 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:36.419923 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:36.419933 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:36.420299 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:36.421265 (+ 966us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:15:36.421271 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:36.421338 (+ 67us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:15:36.421476 (+ 138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:36.421991 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:36.422006 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:36.422023 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 18:15:36.422242 (+ 219us) client_negotiation.cc:240] Negotiation successful
0807 18:15:36.422787 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":356,"threads_started":1}
I20260807 18:15:36.425799 31592 heartbeater.cc:347] Connected to a master server at 127.17.110.62:46645
I20260807 18:15:36.426141 31592 heartbeater.cc:464] Registering TS with master...
I20260807 18:15:36.427214 31592 heartbeater.cc:511] Master 127.17.110.62:46645 requested a full tablet report, sending...
I20260807 18:15:36.428864 31117 ts_manager.cc:194] Registered new tserver with Master: d600c800abc74751b2cfd20d5717cb1b (127.17.110.2:46823)
I20260807 18:15:36.429986 31117 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.110.2:53637
W20260807 18:15:36.786381 31599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:36.786765 31599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:36.786888 31599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:36.794622 31599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:36.794854 31599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:36.794979 31599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.3
I20260807 18:15:36.804721 31599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:38665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:46645
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.3
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:15:36.806063 31599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:36.808215 31599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:36.823351 31606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:36.823477 31609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:36.823978 31605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:36.825392 31599 server_base.cc:1034] running on GCE node
I20260807 18:15:36.826174 31599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:36.827385 31599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:36.828656 31599 hybrid_clock.cc:648] HybridClock initialized: now 1786126536828602 us; error 50 us; skew 500 ppm
I20260807 18:15:36.829015 31599 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:15:36.831812 31599 webserver.cc:460] Webserver started at http://127.17.110.3:36273/ using document root <none> and password file <none>
I20260807 18:15:36.832659 31599 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:36.832808 31599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:36.833280 31599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:15:36.836596 31599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/data/instance:
uuid: "094367933ae84a2f8059c6080d07ea7c"
format_stamp: "Formatted at 2026-08-07 18:15:36 on dist-test-slave-39l8"
server_key: "3bade9259763a2ca0d08bd1191e30e33"
server_key_iv: "194263746c1a2085331f37f384bc9581"
server_key_version: "encryptionkey@0"
I20260807 18:15:36.837565 31599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance:
uuid: "094367933ae84a2f8059c6080d07ea7c"
format_stamp: "Formatted at 2026-08-07 18:15:36 on dist-test-slave-39l8"
server_key: "3bade9259763a2ca0d08bd1191e30e33"
server_key_iv: "194263746c1a2085331f37f384bc9581"
server_key_version: "encryptionkey@0"
I20260807 18:15:36.843652 31599 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 18:15:36.848081 31616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:36.849898 31599 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260807 18:15:36.850077 31599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/wal
uuid: "094367933ae84a2f8059c6080d07ea7c"
format_stamp: "Formatted at 2026-08-07 18:15:36 on dist-test-slave-39l8"
server_key: "3bade9259763a2ca0d08bd1191e30e33"
server_key_iv: "194263746c1a2085331f37f384bc9581"
server_key_version: "encryptionkey@0"
I20260807 18:15:36.850291 31599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:36.883504 31599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:36.884555 31599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:36.884881 31599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:36.885963 31599 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:15:36.887722 31599 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:15:36.887827 31599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:36.887912 31599 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:15:36.887967 31599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:36.901661 31401 heartbeater.cc:503] Master 127.17.110.62:46645 was elected leader, sending a full tablet report...
I20260807 18:15:36.927454 31599 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.3:37213
I20260807 18:15:36.927603 31779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.3:37213 every 8 connection(s)
I20260807 18:15:36.928905 31599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
I20260807 18:15:36.931216 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 31599
I20260807 18:15:36.931419 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance
I20260807 18:15:36.932142 17848 external_mini_cluster.cc:1428] Setting key 1187c30fbd4988e02722973bbbc92419
I20260807 18:15:36.943342 31784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:36.931990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:42377 (local address 127.17.110.62:46645)
0807 18:15:36.932506 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:36.932511 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:36.934211 (+ 1700us) server_negotiation.cc:407] Connection header received
0807 18:15:36.936217 (+ 2006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:36.936221 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:36.936297 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:36.936392 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:36.937618 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:36.938450 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:36.940341 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:36.940778 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:36.940891 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:36.942443 (+ 1552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:36.942448 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:36.942451 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:15:36.942474 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:36.942522 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:15:36.942534 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:36.942613 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:36.942943 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 18:15:36.943039 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":287,"threads_started":1}
I20260807 18:15:36.944435 31785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:36.932823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:46645 (local address 127.17.110.3:42377)
0807 18:15:36.934013 (+ 1190us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:36.934077 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:36.935874 (+ 1797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:36.936630 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:36.936638 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:36.936737 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:36.937433 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:36.937452 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:36.938618 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:36.938623 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:36.940128 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:36.940144 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:36.941004 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:36.941009 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:36.941241 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:36.942070 (+ 829us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:15:36.942072 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:36.942118 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:15:36.942212 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:36.942627 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:36.942635 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:36.942641 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 18:15:36.942913 (+ 272us) client_negotiation.cc:240] Negotiation successful
0807 18:15:36.943272 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":383,"threads_started":1}
I20260807 18:15:36.946070 31781 heartbeater.cc:347] Connected to a master server at 127.17.110.62:46645
I20260807 18:15:36.946439 31781 heartbeater.cc:464] Registering TS with master...
I20260807 18:15:36.947461 31781 heartbeater.cc:511] Master 127.17.110.62:46645 requested a full tablet report, sending...
I20260807 18:15:36.949047 31117 ts_manager.cc:194] Registered new tserver with Master: 094367933ae84a2f8059c6080d07ea7c (127.17.110.3:37213)
I20260807 18:15:36.949980 31117 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.110.3:42377
I20260807 18:15:36.961408 17848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 18:15:36.968408 31784 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:36.966642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55200 (local address 127.17.110.62:46645)
0807 18:15:36.966879 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:36.966884 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:36.967338 (+ 454us) server_negotiation.cc:407] Connection header received
0807 18:15:36.967488 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:36.967493 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:36.967568 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:36.967672 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:36.968078 (+ 406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55200: BlockingRecv error: recv got EOF from 127.0.0.1:55200 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 18:15:36.969501 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 31200
I20260807 18:15:36.979152 31797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:36.970777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34134 (local address 127.17.110.1:45891)
0807 18:15:36.971457 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:36.971466 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:36.971498 (+ 32us) server_negotiation.cc:407] Connection header received
0807 18:15:36.971564 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:36.971571 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:36.971865 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:36.972050 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:36.972526 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:36.973697 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:36.975218 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:36.975913 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:36.976034 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:36.977376 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:36.977386 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:36.977402 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:15:36.978089 (+ 687us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:36.978151 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:15:36.978177 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:36.978307 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:36.978482 (+ 175us) server_negotiation.cc:299] Negotiation successful
0807 18:15:36.978670 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":339,"threads_started":1}
I20260807 18:15:36.980618 31395 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:37.067787 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 31200
I20260807 18:15:37.091332 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 31407
I20260807 18:15:37.100122 31800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:37.092377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50722 (local address 127.17.110.2:46823)
0807 18:15:37.093072 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:37.093080 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:37.093114 (+ 34us) server_negotiation.cc:407] Connection header received
0807 18:15:37.093196 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:37.093202 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:37.093447 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:37.093611 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:37.093988 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:37.095242 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:37.096533 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:37.097210 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:37.097339 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:37.098508 (+ 1169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:37.098517 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:37.098532 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:15:37.099289 (+ 757us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:37.099341 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:15:37.099363 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:37.099480 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:37.099656 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 18:15:37.099780 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":374,"threads_started":1}
I20260807 18:15:37.101511 31586 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:37.190594 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 31407
I20260807 18:15:37.218060 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 31599
I20260807 18:15:37.227183 31807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:37.219173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52070 (local address 127.17.110.3:37213)
0807 18:15:37.219852 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:37.219860 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:37.219892 (+ 32us) server_negotiation.cc:407] Connection header received
0807 18:15:37.219957 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:37.219963 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:37.220226 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:37.220365 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:37.220795 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:37.221863 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:37.223360 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:37.224041 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:37.224171 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:37.225487 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:37.225496 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:37.225511 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:15:37.226199 (+ 688us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:37.226287 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:15:37.226312 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:37.226453 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:37.226581 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 18:15:37.226788 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":350,"threads_started":1}
I20260807 18:15:37.228695 31775 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:37.316167 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 31599
I20260807 18:15:37.343856 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 31079
I20260807 18:15:37.345278 31160 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:37.453028 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 31079
2026-08-07T18:15:37Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2770 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T18:15:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:15:37Z Disabled control of system clock
I20260807 18:15:37.551005 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:44567
--webserver_interface=127.17.110.62
--webserver_port=0
--builtin_ntp_servers=127.17.110.20:34323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:44567
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 18:15:37.916168 31818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:37.916519 31818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:37.916612 31818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:37.924319 31818 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:15:37.924458 31818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:37.924496 31818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:37.924548 31818 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:15:37.924585 31818 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:15:37.934603 31818 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:34323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:44567
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:44567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:15:37.936124 31818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:37.938084 31818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:37.951617 31827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:37.951619 31830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:37.951712 31828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:37.953282 31818 server_base.cc:1034] running on GCE node
I20260807 18:15:37.954330 31818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:37.956075 31818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:37.957387 31818 hybrid_clock.cc:648] HybridClock initialized: now 1786126537957336 us; error 57 us; skew 500 ppm
I20260807 18:15:37.957809 31818 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:15:37.960505 31818 webserver.cc:460] Webserver started at http://127.17.110.62:44231/ using document root <none> and password file <none>
I20260807 18:15:37.961310 31818 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:37.961465 31818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:37.961841 31818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:15:37.965094 31818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "9c3dc57bc00442898229cca8f352897d"
format_stamp: "Formatted at 2026-08-07 18:15:37 on dist-test-slave-39l8"
server_key: "66b32699be8da0d8123b796aa8d0f35b"
server_key_iv: "492ee1f972664499243949ddb37a3783"
server_key_version: "encryptionkey@0"
I20260807 18:15:37.965996 31818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "9c3dc57bc00442898229cca8f352897d"
format_stamp: "Formatted at 2026-08-07 18:15:37 on dist-test-slave-39l8"
server_key: "66b32699be8da0d8123b796aa8d0f35b"
server_key_iv: "492ee1f972664499243949ddb37a3783"
server_key_version: "encryptionkey@0"
I20260807 18:15:37.972055 31818 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260807 18:15:37.976195 31840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:37.977870 31818 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:15:37.978113 31818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "9c3dc57bc00442898229cca8f352897d"
format_stamp: "Formatted at 2026-08-07 18:15:37 on dist-test-slave-39l8"
server_key: "66b32699be8da0d8123b796aa8d0f35b"
server_key_iv: "492ee1f972664499243949ddb37a3783"
server_key_version: "encryptionkey@0"
I20260807 18:15:37.978406 31818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:37.985103 31818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:37.986047 31818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:37.986413 31818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:38.006930 31818 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:44567
I20260807 18:15:38.006951 31908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:44567 every 8 connection(s)
I20260807 18:15:38.008460 31818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:15:38.012265 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 31818
I20260807 18:15:38.012504 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:15:38.013293 31910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:38.013401 17848 external_mini_cluster.cc:1428] Setting key 4c990cb394a78af23811534082fad971
I20260807 18:15:38.026623 31914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:38.016576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43742 (local address 127.17.110.62:44567)
0807 18:15:38.017991 (+ 1415us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:38.018011 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:38.018073 (+ 62us) server_negotiation.cc:407] Connection header received
0807 18:15:38.022252 (+ 4179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:38.022297 (+ 45us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:38.023094 (+ 797us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:38.023612 (+ 518us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:38.024788 (+ 1176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:38.024794 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:38.024808 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:15:38.024824 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:38.024892 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:15:38.024910 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:38.025190 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:38.025349 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 18:15:38.025637 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":752,"thread_start_us":462,"threads_started":1}
I20260807 18:15:38.030117 31910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c3dc57bc00442898229cca8f352897d: Bootstrap starting.
I20260807 18:15:38.033835 31910 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c3dc57bc00442898229cca8f352897d: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:38.035110 31910 log.cc:824] T 00000000000000000000000000000000 P 9c3dc57bc00442898229cca8f352897d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:15:38.038673 31910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c3dc57bc00442898229cca8f352897d: No bootstrap required, opened a new log
I20260807 18:15:38.047224 31910 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c3dc57bc00442898229cca8f352897d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c3dc57bc00442898229cca8f352897d" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 44567 } }
I20260807 18:15:38.047711 31910 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c3dc57bc00442898229cca8f352897d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:38.047816 31910 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c3dc57bc00442898229cca8f352897d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c3dc57bc00442898229cca8f352897d, State: Initialized, Role: FOLLOWER
I20260807 18:15:38.048578 31910 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c3dc57bc00442898229cca8f352897d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c3dc57bc00442898229cca8f352897d" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 44567 } }
I20260807 18:15:38.048950 31910 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c3dc57bc00442898229cca8f352897d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:15:38.049144 31910 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c3dc57bc00442898229cca8f352897d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:15:38.049377 31910 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c3dc57bc00442898229cca8f352897d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:38.052148 31910 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c3dc57bc00442898229cca8f352897d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c3dc57bc00442898229cca8f352897d" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 44567 } }
I20260807 18:15:38.052745 31910 leader_election.cc:302] T 00000000000000000000000000000000 P 9c3dc57bc00442898229cca8f352897d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c3dc57bc00442898229cca8f352897d; no voters:
I20260807 18:15:38.053373 31910 leader_election.cc:288] T 00000000000000000000000000000000 P 9c3dc57bc00442898229cca8f352897d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:15:38.053551 31916 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c3dc57bc00442898229cca8f352897d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:15:38.054347 31916 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c3dc57bc00442898229cca8f352897d [term 1 LEADER]: Becoming Leader. State: Replica: 9c3dc57bc00442898229cca8f352897d, State: Running, Role: LEADER
I20260807 18:15:38.055019 31916 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c3dc57bc00442898229cca8f352897d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c3dc57bc00442898229cca8f352897d" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 44567 } }
I20260807 18:15:38.055750 31910 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c3dc57bc00442898229cca8f352897d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:15:38.058174 31918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c3dc57bc00442898229cca8f352897d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c3dc57bc00442898229cca8f352897d. Latest consensus state: current_term: 1 leader_uuid: "9c3dc57bc00442898229cca8f352897d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c3dc57bc00442898229cca8f352897d" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 44567 } } }
I20260807 18:15:38.058744 31918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c3dc57bc00442898229cca8f352897d [sys.catalog]: This master's current role is: LEADER
I20260807 18:15:38.058174 31917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c3dc57bc00442898229cca8f352897d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c3dc57bc00442898229cca8f352897d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c3dc57bc00442898229cca8f352897d" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 44567 } } }
I20260807 18:15:38.063061 31917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c3dc57bc00442898229cca8f352897d [sys.catalog]: This master's current role is: LEADER
I20260807 18:15:38.063768 31926 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:15:38.069679 31926 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:15:38.080466 31926 catalog_manager.cc:1347] Generated new cluster ID: da597ce602f2455bbd5f6e7df7c68bc5
I20260807 18:15:38.080611 31926 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:15:38.097702 31926 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:15:38.098304 31926 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:15:38.111721 31926 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c3dc57bc00442898229cca8f352897d: Generated new TSK 0
I20260807 18:15:38.113068 31926 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:15:38.128511 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.1:0
--local_ip_for_outbound_sockets=127.17.110.1
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:44567
--builtin_ntp_servers=127.17.110.20:34323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 18:15:38.497816 31940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:38.498201 31940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:38.498370 31940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:38.505954 31940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:38.506166 31940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:38.506350 31940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.1
I20260807 18:15:38.516165 31940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:34323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:44567
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.1
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:15:38.517575 31940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:38.519582 31940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:38.533870 31952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:38.534027 31956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:38.534233 31960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:38.535315 31940 server_base.cc:1034] running on GCE node
I20260807 18:15:38.536094 31940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:38.537245 31940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:38.538503 31940 hybrid_clock.cc:648] HybridClock initialized: now 1786126538538452 us; error 48 us; skew 500 ppm
I20260807 18:15:38.538932 31940 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:15:38.541559 31940 webserver.cc:460] Webserver started at http://127.17.110.1:43177/ using document root <none> and password file <none>
I20260807 18:15:38.542371 31940 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:38.542520 31940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:38.542954 31940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:15:38.546026 31940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/data/instance:
uuid: "d4fa6b27f34c44b1b2976f54348b8fba"
format_stamp: "Formatted at 2026-08-07 18:15:38 on dist-test-slave-39l8"
server_key: "fe82903eb94f23f58aeb82f72fd6a347"
server_key_iv: "687e1ebcb157b9743870a7d58e092596"
server_key_version: "encryptionkey@0"
I20260807 18:15:38.547019 31940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance:
uuid: "d4fa6b27f34c44b1b2976f54348b8fba"
format_stamp: "Formatted at 2026-08-07 18:15:38 on dist-test-slave-39l8"
server_key: "fe82903eb94f23f58aeb82f72fd6a347"
server_key_iv: "687e1ebcb157b9743870a7d58e092596"
server_key_version: "encryptionkey@0"
I20260807 18:15:38.553090 31940 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 18:15:38.557695 31969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:38.559695 31940 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 18:15:38.559979 31940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/wal
uuid: "d4fa6b27f34c44b1b2976f54348b8fba"
format_stamp: "Formatted at 2026-08-07 18:15:38 on dist-test-slave-39l8"
server_key: "fe82903eb94f23f58aeb82f72fd6a347"
server_key_iv: "687e1ebcb157b9743870a7d58e092596"
server_key_version: "encryptionkey@0"
I20260807 18:15:38.560329 31940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:38.567580 31940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:38.568580 31940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:38.568912 31940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:38.569993 31940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:15:38.571928 31940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:15:38.572093 31940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:38.572244 31940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:15:38.572342 31940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:38.611196 31940 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.1:39995
I20260807 18:15:38.611388 32117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.1:39995 every 8 connection(s)
I20260807 18:15:38.612841 31940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
I20260807 18:15:38.622820 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 31940
I20260807 18:15:38.622834 32120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:38.615998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:54541 (local address 127.17.110.62:44567)
0807 18:15:38.616525 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:38.616530 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:38.617952 (+ 1422us) server_negotiation.cc:407] Connection header received
0807 18:15:38.620023 (+ 2071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:38.620029 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:38.620119 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:38.620262 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:38.621842 (+ 1580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:38.621849 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:38.621854 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:15:38.621887 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:38.621939 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:15:38.621952 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:38.622118 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:38.622349 (+ 231us) server_negotiation.cc:299] Negotiation successful
0807 18:15:38.622435 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":269,"threads_started":1}
I20260807 18:15:38.623083 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance
I20260807 18:15:38.623759 17848 external_mini_cluster.cc:1428] Setting key d4a8ba14936509dfa0c1a8dd05fc896d
I20260807 18:15:38.624039 32121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:38.616688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:44567 (local address 127.17.110.1:54541)
0807 18:15:38.617749 (+ 1061us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:38.617804 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:38.619687 (+ 1883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:38.620544 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:38.620553 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:38.620660 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:38.621479 (+ 819us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:15:38.621482 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:38.621526 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:15:38.621628 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:38.622067 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:38.622080 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:38.622090 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 18:15:38.622451 (+ 361us) client_negotiation.cc:240] Negotiation successful
0807 18:15:38.622815 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":309,"threads_started":1}
I20260807 18:15:38.625952 32118 heartbeater.cc:347] Connected to a master server at 127.17.110.62:44567
I20260807 18:15:38.626332 32118 heartbeater.cc:464] Registering TS with master...
I20260807 18:15:38.627120 32118 heartbeater.cc:511] Master 127.17.110.62:44567 requested a full tablet report, sending...
I20260807 18:15:38.627389 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.2:0
--local_ip_for_outbound_sockets=127.17.110.2
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:44567
--builtin_ntp_servers=127.17.110.20:34323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 18:15:38.629576 31859 ts_manager.cc:194] Registered new tserver with Master: d4fa6b27f34c44b1b2976f54348b8fba (127.17.110.1:39995)
W20260807 18:15:38.991807 32122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:38.992156 32122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:38.992251 32122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:39.000207 32122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:39.000344 32122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:39.000461 32122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.2
I20260807 18:15:39.010370 32122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:34323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:44567
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.2
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:15:39.011909 32122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:39.013873 32122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:39.028726 32128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:39.028697 32131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:39.029246 32133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:39.029469 32122 server_base.cc:1034] running on GCE node
I20260807 18:15:39.030194 32122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:39.031293 32122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:39.032514 32122 hybrid_clock.cc:648] HybridClock initialized: now 1786126539032445 us; error 67 us; skew 500 ppm
I20260807 18:15:39.032939 32122 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:15:39.035507 32122 webserver.cc:460] Webserver started at http://127.17.110.2:37949/ using document root <none> and password file <none>
I20260807 18:15:39.036334 32122 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:39.036470 32122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:39.036856 32122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:15:39.040218 32122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/data/instance:
uuid: "c402130e367945b8a72c9e783a3b1bdf"
format_stamp: "Formatted at 2026-08-07 18:15:39 on dist-test-slave-39l8"
server_key: "b92a326c2f52a25832b5da25973e295e"
server_key_iv: "9d832525ea46c8b0a86f50d0268ba29b"
server_key_version: "encryptionkey@0"
I20260807 18:15:39.041152 32122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance:
uuid: "c402130e367945b8a72c9e783a3b1bdf"
format_stamp: "Formatted at 2026-08-07 18:15:39 on dist-test-slave-39l8"
server_key: "b92a326c2f52a25832b5da25973e295e"
server_key_iv: "9d832525ea46c8b0a86f50d0268ba29b"
server_key_version: "encryptionkey@0"
I20260807 18:15:39.047315 32122 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 18:15:39.051733 32145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:39.053591 32122 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:15:39.053840 32122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/wal
uuid: "c402130e367945b8a72c9e783a3b1bdf"
format_stamp: "Formatted at 2026-08-07 18:15:39 on dist-test-slave-39l8"
server_key: "b92a326c2f52a25832b5da25973e295e"
server_key_iv: "9d832525ea46c8b0a86f50d0268ba29b"
server_key_version: "encryptionkey@0"
I20260807 18:15:39.054124 32122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:39.061034 32122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:39.062155 32122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:39.062467 32122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:39.063593 32122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:15:39.065238 32122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:15:39.065372 32122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:39.065522 32122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:15:39.065615 32122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:39.121014 32122 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.2:45801
I20260807 18:15:39.121093 32305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.2:45801 every 8 connection(s)
I20260807 18:15:39.123095 32122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
I20260807 18:15:39.129962 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 32122
I20260807 18:15:39.130198 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance
I20260807 18:15:39.130697 17848 external_mini_cluster.cc:1428] Setting key 9300184605788872189ff00fbd140374
I20260807 18:15:39.134209 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.3:0
--local_ip_for_outbound_sockets=127.17.110.3
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:44567
--builtin_ntp_servers=127.17.110.20:34323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 18:15:39.136843 32309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:39.127077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:44567 (local address 127.17.110.2:39791)
0807 18:15:39.128448 (+ 1371us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:39.128527 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:39.131501 (+ 2974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:39.132510 (+ 1009us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:39.132526 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:39.132697 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:39.133600 (+ 903us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:15:39.133606 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:39.133675 (+ 69us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:15:39.133831 (+ 156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:39.134486 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:39.134503 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:39.134515 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 18:15:39.134896 (+ 381us) client_negotiation.cc:240] Negotiation successful
0807 18:15:39.135278 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":357,"threads_started":1}
I20260807 18:15:39.137915 32308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:39.126233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:39791 (local address 127.17.110.62:44567)
0807 18:15:39.126789 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:39.126797 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:39.130910 (+ 4113us) server_negotiation.cc:407] Connection header received
0807 18:15:39.131917 (+ 1007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:39.131923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:39.132000 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:39.132137 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:39.134099 (+ 1962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:39.134107 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:39.134125 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:15:39.134154 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:39.134227 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:15:39.134250 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:39.134414 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:39.137431 (+ 3017us) server_negotiation.cc:299] Negotiation successful
0807 18:15:39.137529 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":244,"threads_started":1}
I20260807 18:15:39.139312 32306 heartbeater.cc:347] Connected to a master server at 127.17.110.62:44567
I20260807 18:15:39.139707 32306 heartbeater.cc:464] Registering TS with master...
I20260807 18:15:39.140550 32306 heartbeater.cc:511] Master 127.17.110.62:44567 requested a full tablet report, sending...
I20260807 18:15:39.142634 31859 ts_manager.cc:194] Registered new tserver with Master: c402130e367945b8a72c9e783a3b1bdf (127.17.110.2:45801)
W20260807 18:15:39.518167 32310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:39.518529 32310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:39.518698 32310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:39.526981 32310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:39.527128 32310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:39.527343 32310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.3
I20260807 18:15:39.537231 32310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:34323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:44567
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.3
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:15:39.538678 32310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:39.540943 32310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:39.555902 32318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:39.555858 32319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:39.555979 32321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:39.557380 32310 server_base.cc:1034] running on GCE node
I20260807 18:15:39.558359 32310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:39.559751 32310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:39.561010 32310 hybrid_clock.cc:648] HybridClock initialized: now 1786126539560943 us; error 89 us; skew 500 ppm
I20260807 18:15:39.561386 32310 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:15:39.564215 32310 webserver.cc:460] Webserver started at http://127.17.110.3:33041/ using document root <none> and password file <none>
I20260807 18:15:39.565167 32310 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:39.565313 32310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:39.565726 32310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:15:39.569217 32310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/data/instance:
uuid: "13f4a5d2f8554097ac02351eedb57b2f"
format_stamp: "Formatted at 2026-08-07 18:15:39 on dist-test-slave-39l8"
server_key: "c1fe273569b0af8304544a0be534c817"
server_key_iv: "9329fd022c63a023b5afb97cb601c383"
server_key_version: "encryptionkey@0"
I20260807 18:15:39.570293 32310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance:
uuid: "13f4a5d2f8554097ac02351eedb57b2f"
format_stamp: "Formatted at 2026-08-07 18:15:39 on dist-test-slave-39l8"
server_key: "c1fe273569b0af8304544a0be534c817"
server_key_iv: "9329fd022c63a023b5afb97cb601c383"
server_key_version: "encryptionkey@0"
I20260807 18:15:39.576984 32310 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260807 18:15:39.583352 32331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:39.585212 32310 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.002s
I20260807 18:15:39.585486 32310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/wal
uuid: "13f4a5d2f8554097ac02351eedb57b2f"
format_stamp: "Formatted at 2026-08-07 18:15:39 on dist-test-slave-39l8"
server_key: "c1fe273569b0af8304544a0be534c817"
server_key_iv: "9329fd022c63a023b5afb97cb601c383"
server_key_version: "encryptionkey@0"
I20260807 18:15:39.585798 32310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:39.593580 32310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:39.594666 32310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:39.595090 32310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:39.596276 32310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:15:39.598067 32310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:15:39.598220 32310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:39.598384 32310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:15:39.598489 32310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:39.632896 32118 heartbeater.cc:503] Master 127.17.110.62:44567 was elected leader, sending a full tablet report...
I20260807 18:15:39.640586 32310 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.3:43801
I20260807 18:15:39.640650 32494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.3:43801 every 8 connection(s)
I20260807 18:15:39.642081 32310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
I20260807 18:15:39.650380 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 32310
I20260807 18:15:39.650627 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance
I20260807 18:15:39.651365 17848 external_mini_cluster.cc:1428] Setting key ebd40d1f439a85a92e7e6021cf1ee23d
I20260807 18:15:39.651507 32498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:39.645060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:44641 (local address 127.17.110.62:44567)
0807 18:15:39.645541 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:39.645546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:39.646958 (+ 1412us) server_negotiation.cc:407] Connection header received
0807 18:15:39.648793 (+ 1835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:39.648797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:39.648862 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:39.648972 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:39.650579 (+ 1607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:39.650585 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:39.650591 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:15:39.650619 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:39.650678 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:15:39.650695 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:39.650888 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:39.651160 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 18:15:39.651223 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":258,"threads_started":1}
I20260807 18:15:39.652825 32500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:39.645709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:44567 (local address 127.17.110.3:44641)
0807 18:15:39.646690 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:39.646834 (+ 144us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:39.648483 (+ 1649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:39.649245 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:39.649255 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:39.649361 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:39.650127 (+ 766us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:15:39.650133 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:39.650191 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:15:39.650325 (+ 134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:39.650910 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:39.650922 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:39.650930 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 18:15:39.651247 (+ 317us) client_negotiation.cc:240] Negotiation successful
0807 18:15:39.651558 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":280,"threads_started":1}
I20260807 18:15:39.654433 32495 heartbeater.cc:347] Connected to a master server at 127.17.110.62:44567
I20260807 18:15:39.654826 32495 heartbeater.cc:464] Registering TS with master...
I20260807 18:15:39.655480 32495 heartbeater.cc:511] Master 127.17.110.62:44567 requested a full tablet report, sending...
I20260807 18:15:39.657280 31861 ts_manager.cc:194] Registered new tserver with Master: 13f4a5d2f8554097ac02351eedb57b2f (127.17.110.3:43801)
I20260807 18:15:39.667698 17848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 18:15:39.674638 32498 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:15:39.672756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43746 (local address 127.17.110.62:44567)
0807 18:15:39.672952 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:39.672957 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:39.673555 (+ 598us) server_negotiation.cc:407] Connection header received
0807 18:15:39.673712 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:39.673716 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:39.673793 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:39.673894 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:39.674324 (+ 430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43746: BlockingRecv error: recv got EOF from 127.0.0.1:43746 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 18:15:39.675325 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 31940
I20260807 18:15:39.679617 32513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:39.676690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43156 (local address 127.17.110.1:39995)
0807 18:15:39.677329 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:39.677338 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:39.677367 (+ 29us) server_negotiation.cc:407] Connection header received
0807 18:15:39.677431 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:39.677437 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:39.677670 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:39.677824 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:39.678905 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:39.678913 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:39.678926 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:15:39.678941 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:39.678995 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:15:39.679011 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:39.679166 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:39.679217 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 18:15:39.679317 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":314,"threads_started":1}
I20260807 18:15:39.681066 32113 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:39.770273 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 31940
I20260807 18:15:39.800467 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 32122
I20260807 18:15:39.805536 32515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:39.801779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48776 (local address 127.17.110.2:45801)
0807 18:15:39.802411 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:39.802419 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:39.802451 (+ 32us) server_negotiation.cc:407] Connection header received
0807 18:15:39.802516 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:39.802521 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:39.802925 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:39.803091 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:39.804274 (+ 1183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:39.804284 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:39.804493 (+ 209us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:15:39.804534 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:39.804601 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:15:39.804627 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:39.804764 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:39.805097 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 18:15:39.805225 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":322,"threads_started":1}
I20260807 18:15:39.807241 32299 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:39.903519 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 32122
I20260807 18:15:39.935807 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 32310
I20260807 18:15:39.940929 32517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:39.937473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48418 (local address 127.17.110.3:43801)
0807 18:15:39.938238 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:39.938246 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:39.938283 (+ 37us) server_negotiation.cc:407] Connection header received
0807 18:15:39.938351 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:39.938358 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:39.938590 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:39.938857 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:39.939966 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:39.939973 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:39.939985 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:15:39.939998 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:39.940041 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:15:39.940059 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:39.940210 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:39.940384 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 18:15:39.940532 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":412,"threads_started":1}
I20260807 18:15:39.942462 32489 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:40.034632 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 32310
I20260807 18:15:40.064635 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 31818
I20260807 18:15:40.066234 31903 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:40.182902 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 31818
2026-08-07T18:15:40Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2728 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:15:40 dist-test-slave-39l8 krb5kdc[32525](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:15:40 dist-test-slave-39l8 krb5kdc[32525](info): set up 2 sockets
Aug 07 18:15:40 dist-test-slave-39l8 krb5kdc[32525](info): commencing operation
krb5kdc: starting...
W20260807 18:15:41.568696 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.334s user 0.005s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:15:41 dist-test-slave-39l8 krb5kdc[32525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:15:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:15:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.110.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
I20260807 18:15:41.887470 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:41333
--webserver_interface=127.17.110.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:42133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:41333
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:15:42.259341 32545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:42.259676 32545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:42.259829 32545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:42.267432 32545 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:15:42.267568 32545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:42.267616 32545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:42.267757 32545 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:15:42.267817 32545 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:15:42.278859 32545 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:42133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:41333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:41333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:15:42.280496 32545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:42.282083 32545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:42.295039 32554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:42.295153 32551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:42.295085 32552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:42.296106 32545 server_base.cc:1034] running on GCE node
I20260807 18:15:42.296921 32545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:42.298417 32545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:42.299680 32545 hybrid_clock.cc:648] HybridClock initialized: now 1786126542299595 us; error 69 us; skew 500 ppm
I20260807 18:15:42.300204 32545 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:15:42 dist-test-slave-39l8 krb5kdc[32525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126542, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:42.304140 32545 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:15:42.305910 32545 webserver.cc:460] Webserver started at http://127.17.110.62:39525/ using document root <none> and password file <none>
I20260807 18:15:42.306692 32545 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:42.306892 32545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:42.307204 32545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:15:42.310758 32545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "a63e7e43509844c194217b25cc2df981"
format_stamp: "Formatted at 2026-08-07 18:15:42 on dist-test-slave-39l8"
server_key: "9fe4adcdd3bb6603d34e898fe1a3fe4c"
server_key_iv: "bb2886ec12c9d78a60bbc291ec5d84c6"
server_key_version: "encryptionkey@0"
I20260807 18:15:42.311694 32545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "a63e7e43509844c194217b25cc2df981"
format_stamp: "Formatted at 2026-08-07 18:15:42 on dist-test-slave-39l8"
server_key: "9fe4adcdd3bb6603d34e898fe1a3fe4c"
server_key_iv: "bb2886ec12c9d78a60bbc291ec5d84c6"
server_key_version: "encryptionkey@0"
I20260807 18:15:42.317952 32545 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 18:15:42.322892 32563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:42.324621 32545 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:15:42.324874 32545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "a63e7e43509844c194217b25cc2df981"
format_stamp: "Formatted at 2026-08-07 18:15:42 on dist-test-slave-39l8"
server_key: "9fe4adcdd3bb6603d34e898fe1a3fe4c"
server_key_iv: "bb2886ec12c9d78a60bbc291ec5d84c6"
server_key_version: "encryptionkey@0"
I20260807 18:15:42.325125 32545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:42.360040 32545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:42.366648 32545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:42.367086 32545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:42.388506 32545 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:41333
I20260807 18:15:42.388538 32644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:41333 every 8 connection(s)
I20260807 18:15:42.390046 32545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:15:42.392446 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 32545
I20260807 18:15:42.392766 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:15:42.393465 17848 external_mini_cluster.cc:1428] Setting key b5ce87e7f9914c29f964a3a5cb89d466
I20260807 18:15:42.394440 32647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:42.407909 32647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a63e7e43509844c194217b25cc2df981: Bootstrap starting.
Aug 07 18:15:42 dist-test-slave-39l8 krb5kdc[32525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:15:42.411814 32647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a63e7e43509844c194217b25cc2df981: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:42.413046 32647 log.cc:824] T 00000000000000000000000000000000 P a63e7e43509844c194217b25cc2df981: Log is configured to *not* fsync() on all Append() calls
I20260807 18:15:42.416318 32647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a63e7e43509844c194217b25cc2df981: No bootstrap required, opened a new log
I20260807 18:15:42.418292 32650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:42.395716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58206 (local address 127.17.110.62:41333)
0807 18:15:42.396596 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:42.396609 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:42.396652 (+ 43us) server_negotiation.cc:407] Connection header received
0807 18:15:42.398225 (+ 1573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:42.398248 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:42.398775 (+ 527us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:42.399296 (+ 521us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:42.400954 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:42.402337 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:42.403777 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:42.404543 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:42.404763 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:42.409434 (+ 4671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:42.409444 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:42.409464 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:42.409520 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:42.413454 (+ 3934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:42.414376 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:42.414384 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:42.414392 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:42.414610 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:42.415106 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:42.415110 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:42.415112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:42.415711 (+ 599us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:42.416124 (+ 413us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:42.416598 (+ 474us) server_negotiation.cc:299] Negotiation successful
0807 18:15:42.417143 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":389,"threads_started":1}
I20260807 18:15:42.425036 32647 raft_consensus.cc:348] T 00000000000000000000000000000000 P a63e7e43509844c194217b25cc2df981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a63e7e43509844c194217b25cc2df981" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 41333 } }
I20260807 18:15:42.425532 32647 raft_consensus.cc:374] T 00000000000000000000000000000000 P a63e7e43509844c194217b25cc2df981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:42.425670 32647 raft_consensus.cc:729] T 00000000000000000000000000000000 P a63e7e43509844c194217b25cc2df981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a63e7e43509844c194217b25cc2df981, State: Initialized, Role: FOLLOWER
I20260807 18:15:42.426450 32647 consensus_queue.cc:260] T 00000000000000000000000000000000 P a63e7e43509844c194217b25cc2df981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a63e7e43509844c194217b25cc2df981" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 41333 } }
I20260807 18:15:42.426790 32647 raft_consensus.cc:388] T 00000000000000000000000000000000 P a63e7e43509844c194217b25cc2df981 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:15:42.426973 32647 raft_consensus.cc:482] T 00000000000000000000000000000000 P a63e7e43509844c194217b25cc2df981 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:15:42.427198 32647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a63e7e43509844c194217b25cc2df981 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:42.430210 32647 raft_consensus.cc:504] T 00000000000000000000000000000000 P a63e7e43509844c194217b25cc2df981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a63e7e43509844c194217b25cc2df981" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 41333 } }
I20260807 18:15:42.430970 32647 leader_election.cc:302] T 00000000000000000000000000000000 P a63e7e43509844c194217b25cc2df981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a63e7e43509844c194217b25cc2df981; no voters:
I20260807 18:15:42.431648 32647 leader_election.cc:288] T 00000000000000000000000000000000 P a63e7e43509844c194217b25cc2df981 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:15:42.431890 32652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a63e7e43509844c194217b25cc2df981 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:15:42.432798 32652 raft_consensus.cc:686] T 00000000000000000000000000000000 P a63e7e43509844c194217b25cc2df981 [term 1 LEADER]: Becoming Leader. State: Replica: a63e7e43509844c194217b25cc2df981, State: Running, Role: LEADER
I20260807 18:15:42.433506 32652 consensus_queue.cc:237] T 00000000000000000000000000000000 P a63e7e43509844c194217b25cc2df981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a63e7e43509844c194217b25cc2df981" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 41333 } }
I20260807 18:15:42.434319 32647 sys_catalog.cc:564] T 00000000000000000000000000000000 P a63e7e43509844c194217b25cc2df981 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:15:42.436933 32653 sys_catalog.cc:455] T 00000000000000000000000000000000 P a63e7e43509844c194217b25cc2df981 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a63e7e43509844c194217b25cc2df981" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a63e7e43509844c194217b25cc2df981" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 41333 } } }
I20260807 18:15:42.436962 32655 sys_catalog.cc:455] T 00000000000000000000000000000000 P a63e7e43509844c194217b25cc2df981 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a63e7e43509844c194217b25cc2df981. Latest consensus state: current_term: 1 leader_uuid: "a63e7e43509844c194217b25cc2df981" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a63e7e43509844c194217b25cc2df981" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 41333 } } }
I20260807 18:15:42.437461 32653 sys_catalog.cc:458] T 00000000000000000000000000000000 P a63e7e43509844c194217b25cc2df981 [sys.catalog]: This master's current role is: LEADER
I20260807 18:15:42.437489 32655 sys_catalog.cc:458] T 00000000000000000000000000000000 P a63e7e43509844c194217b25cc2df981 [sys.catalog]: This master's current role is: LEADER
I20260807 18:15:42.438619 32666 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:15:42.444422 32666 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:15:42.455271 32666 catalog_manager.cc:1347] Generated new cluster ID: 0dd7864a3bf2497fa2dbedd8f935ea9d
I20260807 18:15:42.455425 32666 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:15:42.475164 32666 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:15:42.476914 32666 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:15:42.487057 32666 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a63e7e43509844c194217b25cc2df981: Generated new TSK 0
I20260807 18:15:42.488190 32666 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.110.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
I20260807 18:15:42.562569 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.1:0
--local_ip_for_outbound_sockets=127.17.110.1
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:41333
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:42133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:15:42.945302 32683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:42.945617 32683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:42.945771 32683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:42.953754 32683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:42.953896 32683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:42.954109 32683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.1
I20260807 18:15:42.964685 32683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:42133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:41333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.1
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:15:42.966192 32683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:42.967633 32683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:42.981858 32692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:42.981937 32691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:42.981956 32694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:42.983357 32683 server_base.cc:1034] running on GCE node
I20260807 18:15:42.984006 32683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:42.985018 32683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:42.986289 32683 hybrid_clock.cc:648] HybridClock initialized: now 1786126542986235 us; error 43 us; skew 500 ppm
I20260807 18:15:42.986780 32683 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:15:42 dist-test-slave-39l8 krb5kdc[32525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126542, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:42.990599 32683 init.cc:377] Logged in from keytab as kudu/127.17.110.1@KRBTEST.COM (short username kudu)
I20260807 18:15:42.992487 32683 webserver.cc:460] Webserver started at http://127.17.110.1:40161/ using document root <none> and password file <none>
I20260807 18:15:42.993327 32683 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:42.993489 32683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:42.993830 32683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:15:42.997083 32683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/data/instance:
uuid: "a45ecaf897aa437a99545955c113d3db"
format_stamp: "Formatted at 2026-08-07 18:15:42 on dist-test-slave-39l8"
server_key: "5481c5312ce21b571e0a1738a869de3b"
server_key_iv: "36f5de342ed5168edd2f16f3f93d8259"
server_key_version: "encryptionkey@0"
I20260807 18:15:42.997958 32683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance:
uuid: "a45ecaf897aa437a99545955c113d3db"
format_stamp: "Formatted at 2026-08-07 18:15:42 on dist-test-slave-39l8"
server_key: "5481c5312ce21b571e0a1738a869de3b"
server_key_iv: "36f5de342ed5168edd2f16f3f93d8259"
server_key_version: "encryptionkey@0"
I20260807 18:15:43.003885 32683 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 18:15:43.007930 32703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:43.009472 32683 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:15:43.009693 32683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/wal
uuid: "a45ecaf897aa437a99545955c113d3db"
format_stamp: "Formatted at 2026-08-07 18:15:42 on dist-test-slave-39l8"
server_key: "5481c5312ce21b571e0a1738a869de3b"
server_key_iv: "36f5de342ed5168edd2f16f3f93d8259"
server_key_version: "encryptionkey@0"
I20260807 18:15:43.009945 32683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:43.027874 32683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:43.033421 32683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:43.033784 32683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:43.035064 32683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:15:43.036653 32683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:15:43.036733 32683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:43.036881 32683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:15:43.036950 32683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:43.072487 32683 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.1:44477
I20260807 18:15:43.072546 395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.1:44477 every 8 connection(s)
I20260807 18:15:43.073961 32683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
I20260807 18:15:43.078192 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 32683
I20260807 18:15:43.078437 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance
I20260807 18:15:43.079217 17848 external_mini_cluster.cc:1428] Setting key 7eabef1b06c8317d34203d128243f411
Aug 07 18:15:43 dist-test-slave-39l8 krb5kdc[32525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126542, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:15:43.098908 401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:43.076882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:56319 (local address 127.17.110.62:41333)
0807 18:15:43.077376 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:43.077381 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:43.078337 (+ 956us) server_negotiation.cc:407] Connection header received
0807 18:15:43.080716 (+ 2379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:43.080721 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:43.080802 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:43.080915 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:43.082864 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:43.084134 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:43.085853 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:43.086562 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:43.086767 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:43.091468 (+ 4701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:43.091479 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:43.091487 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:43.091546 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:43.095067 (+ 3521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:43.096131 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:43.096136 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:43.096144 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:43.096307 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:43.096847 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:43.096851 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:43.096853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:43.097156 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:43.097336 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:43.098187 (+ 851us) server_negotiation.cc:299] Negotiation successful
0807 18:15:43.098470 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":267,"threads_started":1}
I20260807 18:15:43.100077 404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:43.077321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:41333 (local address 127.17.110.1:56319)
0807 18:15:43.078128 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:43.078169 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:43.080326 (+ 2157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:43.081272 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:43.081285 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:43.081973 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:43.082609 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:43.082629 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:43.084339 (+ 1710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:43.084344 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:43.085668 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:43.085682 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:43.086935 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:43.086941 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:43.087260 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:43.088090 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:43.088119 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:43.091150 (+ 3031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:43.095294 (+ 4144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:43.095310 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:43.095334 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:43.095936 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:43.096444 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:43.096449 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:43.096453 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:43.096688 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:43.097311 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:43.097321 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:43.097739 (+ 418us) client_negotiation.cc:769] Sending connection context
0807 18:15:43.098051 (+ 312us) client_negotiation.cc:240] Negotiation successful
0807 18:15:43.098684 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":349,"threads_started":1}
I20260807 18:15:43.102206 397 heartbeater.cc:347] Connected to a master server at 127.17.110.62:41333
I20260807 18:15:43.102631 397 heartbeater.cc:464] Registering TS with master...
I20260807 18:15:43.103873 397 heartbeater.cc:511] Master 127.17.110.62:41333 requested a full tablet report, sending...
I20260807 18:15:43.106971 32585 ts_manager.cc:194] Registered new tserver with Master: a45ecaf897aa437a99545955c113d3db (127.17.110.1:44477)
I20260807 18:15:43.108659 32585 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.1@KRBTEST.COM'} at 127.17.110.1:56319
WARNING: no policy specified for kudu/127.17.110.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
I20260807 18:15:43.154093 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.2:0
--local_ip_for_outbound_sockets=127.17.110.2
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:41333
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:42133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:15:43.516484 408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:43.516793 408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:43.516891 408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:43.524358 408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:43.524487 408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:43.524609 408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.2
I20260807 18:15:43.534783 408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:42133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:41333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.2
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:15:43.536404 408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:43.537823 408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:43.552100 415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:43.552042 418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:43.552392 416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:43.552946 408 server_base.cc:1034] running on GCE node
I20260807 18:15:43.553745 408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:43.555028 408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:43.556273 408 hybrid_clock.cc:648] HybridClock initialized: now 1786126543556207 us; error 62 us; skew 500 ppm
I20260807 18:15:43.556629 408 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:15:43 dist-test-slave-39l8 krb5kdc[32525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126543, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:43.560101 408 init.cc:377] Logged in from keytab as kudu/127.17.110.2@KRBTEST.COM (short username kudu)
I20260807 18:15:43.561818 408 webserver.cc:460] Webserver started at http://127.17.110.2:41865/ using document root <none> and password file <none>
I20260807 18:15:43.562562 408 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:43.562674 408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:43.563026 408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:15:43.566145 408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/data/instance:
uuid: "28556711425a498083dee80e1367e720"
format_stamp: "Formatted at 2026-08-07 18:15:43 on dist-test-slave-39l8"
server_key: "b5f2ee785561a21d774ee127af3cf5d4"
server_key_iv: "12cb290e2af2f5d93993c5f46bc918a8"
server_key_version: "encryptionkey@0"
I20260807 18:15:43.567011 408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance:
uuid: "28556711425a498083dee80e1367e720"
format_stamp: "Formatted at 2026-08-07 18:15:43 on dist-test-slave-39l8"
server_key: "b5f2ee785561a21d774ee127af3cf5d4"
server_key_iv: "12cb290e2af2f5d93993c5f46bc918a8"
server_key_version: "encryptionkey@0"
I20260807 18:15:43.572906 408 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:15:43.577173 428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:43.579044 408 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:15:43.579239 408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/wal
uuid: "28556711425a498083dee80e1367e720"
format_stamp: "Formatted at 2026-08-07 18:15:43 on dist-test-slave-39l8"
server_key: "b5f2ee785561a21d774ee127af3cf5d4"
server_key_iv: "12cb290e2af2f5d93993c5f46bc918a8"
server_key_version: "encryptionkey@0"
I20260807 18:15:43.579454 408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:43.596128 408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:43.600899 408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:43.601287 408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:43.602408 408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:15:43.603942 408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:15:43.604054 408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:43.604209 408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:15:43.604291 408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:43.640798 408 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.2:45705
I20260807 18:15:43.640899 590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.2:45705 every 8 connection(s)
I20260807 18:15:43.642187 408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
I20260807 18:15:43.646356 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 408
I20260807 18:15:43.646651 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance
I20260807 18:15:43.647332 17848 external_mini_cluster.cc:1428] Setting key 9fd8c4527f4b88375d64cb0d8516dffe
Aug 07 18:15:43 dist-test-slave-39l8 krb5kdc[32525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126543, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:15:43.667328 595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:43.645359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:53865 (local address 127.17.110.62:41333)
0807 18:15:43.645933 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:43.645938 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:43.647541 (+ 1603us) server_negotiation.cc:407] Connection header received
0807 18:15:43.649956 (+ 2415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:43.649962 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:43.650060 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:43.650221 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:43.652139 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:43.652954 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:43.654947 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:43.655440 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:43.655599 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:43.660191 (+ 4592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:43.660198 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:43.660202 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:43.660249 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:43.663473 (+ 3224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:43.664489 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:43.664494 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:43.664496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:43.664660 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:43.665218 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:43.665223 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:43.665225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:43.665532 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:43.665720 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:43.666550 (+ 830us) server_negotiation.cc:299] Negotiation successful
0807 18:15:43.666865 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":293,"threads_started":1}
I20260807 18:15:43.668534 596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:43.646183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:41333 (local address 127.17.110.2:53865)
0807 18:15:43.647253 (+ 1070us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:43.647311 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:43.649569 (+ 2258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:43.650509 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:43.650519 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:43.651216 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:43.651897 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:43.651917 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:43.653135 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:43.653149 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:43.654779 (+ 1630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:43.654795 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:43.655735 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:43.655744 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:43.656010 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:43.656837 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:43.656865 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:43.659816 (+ 2951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:43.663716 (+ 3900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:43.663731 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:43.663754 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:43.664303 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:43.664802 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:43.664808 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:43.664812 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:43.665063 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:43.665694 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:43.665704 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:43.666119 (+ 415us) client_negotiation.cc:769] Sending connection context
0807 18:15:43.666427 (+ 308us) client_negotiation.cc:240] Negotiation successful
0807 18:15:43.667138 (+ 711us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":778,"thread_start_us":531,"threads_started":1}
I20260807 18:15:43.670917 591 heartbeater.cc:347] Connected to a master server at 127.17.110.62:41333
I20260807 18:15:43.671314 591 heartbeater.cc:464] Registering TS with master...
I20260807 18:15:43.672375 591 heartbeater.cc:511] Master 127.17.110.62:41333 requested a full tablet report, sending...
I20260807 18:15:43.674247 32585 ts_manager.cc:194] Registered new tserver with Master: 28556711425a498083dee80e1367e720 (127.17.110.2:45705)
I20260807 18:15:43.675331 32585 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.2@KRBTEST.COM'} at 127.17.110.2:53865
WARNING: no policy specified for kudu/127.17.110.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
I20260807 18:15:43.719276 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.3:0
--local_ip_for_outbound_sockets=127.17.110.3
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:41333
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:42133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:15:44.081362 605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:44.081727 605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:44.081898 605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:44.089870 605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:44.090068 605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:44.090260 605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.3
I20260807 18:15:44.100893 605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:42133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:41333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.3
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:15:44.102377 605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:44.103863 605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:15:44.113586 397 heartbeater.cc:503] Master 127.17.110.62:41333 was elected leader, sending a full tablet report...
W20260807 18:15:44.119248 613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:44.119318 614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:44.119268 619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:44.120905 605 server_base.cc:1034] running on GCE node
I20260807 18:15:44.121742 605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:44.122936 605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:44.124254 605 hybrid_clock.cc:648] HybridClock initialized: now 1786126544124161 us; error 105 us; skew 500 ppm
I20260807 18:15:44.124574 605 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:15:44 dist-test-slave-39l8 krb5kdc[32525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126544, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:44.128333 605 init.cc:377] Logged in from keytab as kudu/127.17.110.3@KRBTEST.COM (short username kudu)
I20260807 18:15:44.130390 605 webserver.cc:460] Webserver started at http://127.17.110.3:42893/ using document root <none> and password file <none>
I20260807 18:15:44.131382 605 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:44.131553 605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:44.131902 605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:15:44.135226 605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/data/instance:
uuid: "4fd10fc624034c9b9d25d1cbd38b9c4e"
format_stamp: "Formatted at 2026-08-07 18:15:44 on dist-test-slave-39l8"
server_key: "b92cff89bf0d2ef948e096c8c060e24c"
server_key_iv: "00efca33f3ac0661c3c51d399ac34df0"
server_key_version: "encryptionkey@0"
I20260807 18:15:44.136197 605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance:
uuid: "4fd10fc624034c9b9d25d1cbd38b9c4e"
format_stamp: "Formatted at 2026-08-07 18:15:44 on dist-test-slave-39l8"
server_key: "b92cff89bf0d2ef948e096c8c060e24c"
server_key_iv: "00efca33f3ac0661c3c51d399ac34df0"
server_key_version: "encryptionkey@0"
I20260807 18:15:44.142210 605 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 18:15:44.146186 635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:44.147845 605 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:15:44.148054 605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/wal
uuid: "4fd10fc624034c9b9d25d1cbd38b9c4e"
format_stamp: "Formatted at 2026-08-07 18:15:44 on dist-test-slave-39l8"
server_key: "b92cff89bf0d2ef948e096c8c060e24c"
server_key_iv: "00efca33f3ac0661c3c51d399ac34df0"
server_key_version: "encryptionkey@0"
I20260807 18:15:44.148288 605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:44.172330 605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:44.176913 605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:44.177254 605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:44.178505 605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:15:44.180056 605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:15:44.180173 605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:44.180330 605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:15:44.180403 605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:44.216430 605 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.3:38591
I20260807 18:15:44.216491 795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.3:38591 every 8 connection(s)
I20260807 18:15:44.217764 605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
I20260807 18:15:44.222993 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 605
I20260807 18:15:44.223227 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance
I20260807 18:15:44.223795 17848 external_mini_cluster.cc:1428] Setting key 9306d5a3952704d362cabce2ea4ac866
Aug 07 18:15:44 dist-test-slave-39l8 krb5kdc[32525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126544, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:15:44.240852 799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:44.220553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:47301 (local address 127.17.110.62:41333)
0807 18:15:44.221059 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:44.221064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:44.222050 (+ 986us) server_negotiation.cc:407] Connection header received
0807 18:15:44.224755 (+ 2705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:44.224761 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:44.224886 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:44.225013 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:44.226906 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.227891 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.229548 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.229976 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.230135 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:44.233897 (+ 3762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:44.233902 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:44.233906 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:44.233942 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:44.236906 (+ 2964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:44.237851 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:44.237856 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:44.237859 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:44.238080 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:44.238736 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:44.238744 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:44.238749 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:44.239188 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:44.239326 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:44.240328 (+ 1002us) server_negotiation.cc:299] Negotiation successful
0807 18:15:44.240545 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":257,"threads_started":1}
I20260807 18:15:44.241915 800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:44.221119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:41333 (local address 127.17.110.3:47301)
0807 18:15:44.221864 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:44.221913 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:44.224435 (+ 2522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:44.225283 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:44.225292 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:44.226051 (+ 759us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:44.226671 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:44.226697 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.228067 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.228072 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:44.229397 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:44.229410 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.230222 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.230231 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:44.230495 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:44.231284 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:44.231309 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:44.233639 (+ 2330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:44.237080 (+ 3441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:44.237091 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:44.237112 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:44.237662 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:44.238241 (+ 579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:44.238245 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:44.238250 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:44.238541 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:44.239347 (+ 806us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:44.239357 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:44.239864 (+ 507us) client_negotiation.cc:769] Sending connection context
0807 18:15:44.240176 (+ 312us) client_negotiation.cc:240] Negotiation successful
0807 18:15:44.240739 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":285,"threads_started":1}
I20260807 18:15:44.243546 796 heartbeater.cc:347] Connected to a master server at 127.17.110.62:41333
I20260807 18:15:44.243932 796 heartbeater.cc:464] Registering TS with master...
I20260807 18:15:44.244782 796 heartbeater.cc:511] Master 127.17.110.62:41333 requested a full tablet report, sending...
I20260807 18:15:44.246481 32585 ts_manager.cc:194] Registered new tserver with Master: 4fd10fc624034c9b9d25d1cbd38b9c4e (127.17.110.3:38591)
I20260807 18:15:44.247447 32585 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.3@KRBTEST.COM'} at 127.17.110.3:47301
I20260807 18:15:44.253264 17848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:15:44.273222 799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:44.258453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58214 (local address 127.17.110.62:41333)
0807 18:15:44.258646 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:44.258651 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:44.259041 (+ 390us) server_negotiation.cc:407] Connection header received
0807 18:15:44.259192 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:44.259196 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:44.259269 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:44.259364 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:44.260873 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.261691 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.263478 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.263950 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.264060 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:44.266999 (+ 2939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:44.267005 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:44.267009 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:44.267055 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:44.270120 (+ 3065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:44.271182 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:44.271187 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:44.271190 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:44.271341 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:44.271711 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:44.271714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:44.271716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:44.272090 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:44.272324 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:44.272745 (+ 421us) server_negotiation.cc:299] Negotiation successful
0807 18:15:44.272937 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 18:15:44.278402 32585 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58214:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:15:44.285948 32585 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:15:44.312493 815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:44.300147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:38591 (local address 127.0.0.1:50604)
0807 18:15:44.302371 (+ 2224us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:44.302409 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:44.302590 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:44.303700 (+ 1110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:44.303707 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:44.303737 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:15:44.304221 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:44.304240 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.306413 (+ 2173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.306418 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:44.308910 (+ 2492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:44.308924 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.311087 (+ 2163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.311095 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:44.311399 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:44.311473 (+ 74us) client_negotiation.cc:769] Sending connection context
0807 18:15:44.311648 (+ 175us) client_negotiation.cc:240] Negotiation successful
0807 18:15:44.311892 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1980,"mutex_wait_us":33,"thread_start_us":1811,"threads_started":1}
I20260807 18:15:44.313313 817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:44.300485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50604 (local address 127.17.110.3:38591)
0807 18:15:44.301770 (+ 1285us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:44.301785 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:44.302692 (+ 907us) server_negotiation.cc:407] Connection header received
0807 18:15:44.302839 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:44.302849 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:44.303132 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:44.303338 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:15:44.304454 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.306220 (+ 1766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.309103 (+ 2883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.310805 (+ 1702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.310950 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:44.311235 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:44.312668 (+ 1433us) server_negotiation.cc:299] Negotiation successful
0807 18:15:44.312874 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1039,"thread_start_us":799,"threads_started":1}
I20260807 18:15:44.312594 816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:44.300505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:44477 (local address 127.0.0.1:42236)
0807 18:15:44.303239 (+ 2734us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:44.303261 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:44.303391 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:44.304361 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:44.304366 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:44.304395 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:15:44.304800 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:44.304815 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.306686 (+ 1871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.306762 (+ 76us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:44.309511 (+ 2749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:44.309525 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.311597 (+ 2072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.311607 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:44.311816 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:44.311850 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 18:15:44.311943 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 18:15:44.312122 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2571,"mutex_wait_us":30,"thread_start_us":2432,"threads_started":1}
I20260807 18:15:44.313392 818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:44.301260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42236 (local address 127.17.110.1:44477)
0807 18:15:44.302955 (+ 1695us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:44.302964 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:44.303530 (+ 566us) server_negotiation.cc:407] Connection header received
0807 18:15:44.303639 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:44.303646 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:44.303875 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:44.304071 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:15:44.305021 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.306521 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.309715 (+ 3194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.310963 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.311133 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:44.311408 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:44.312748 (+ 1340us) server_negotiation.cc:299] Negotiation successful
0807 18:15:44.312971 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1470,"thread_start_us":1237,"threads_started":1}
I20260807 18:15:44.316262 820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:44.303410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:45705 (local address 127.0.0.1:35532)
0807 18:15:44.305477 (+ 2067us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:44.305493 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:44.305662 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:44.306261 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:44.306266 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:44.306293 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:15:44.306943 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:44.306960 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.309148 (+ 2188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.309153 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:44.311341 (+ 2188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:44.311354 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.315271 (+ 3917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.315279 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:44.315502 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:44.315540 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 18:15:44.315666 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 18:15:44.315843 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1946,"mutex_wait_us":4,"thread_start_us":1803,"threads_started":1}
I20260807 18:15:44.316313 823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:44.303931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35532 (local address 127.17.110.2:45705)
0807 18:15:44.304992 (+ 1061us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:44.305006 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:44.305618 (+ 612us) server_negotiation.cc:407] Connection header received
0807 18:15:44.305851 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:44.305858 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:44.306056 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:44.306270 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:15:44.307228 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.309012 (+ 1784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.311528 (+ 2516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.314643 (+ 3115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.314836 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:44.315092 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:44.315758 (+ 666us) server_negotiation.cc:299] Negotiation successful
0807 18:15:44.315986 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":861,"thread_start_us":761,"threads_started":1}
I20260807 18:15:44.317966 698 tablet_service.cc:1467] Processing CreateTablet for tablet d8d8d275435d4b368eeaa492996b170e (DEFAULT_TABLE table=test-table [id=2cfba94ba36940d9aafbff9e1dd52341]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:15:44.317961 32767 tablet_service.cc:1467] Processing CreateTablet for tablet d8d8d275435d4b368eeaa492996b170e (DEFAULT_TABLE table=test-table [id=2cfba94ba36940d9aafbff9e1dd52341]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:15:44.319514 32767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8d8d275435d4b368eeaa492996b170e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:44.319783 698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8d8d275435d4b368eeaa492996b170e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:44.321031 503 tablet_service.cc:1467] Processing CreateTablet for tablet d8d8d275435d4b368eeaa492996b170e (DEFAULT_TABLE table=test-table [id=2cfba94ba36940d9aafbff9e1dd52341]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:15:44.322779 503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8d8d275435d4b368eeaa492996b170e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:44.333810 824 tablet_bootstrap.cc:492] T d8d8d275435d4b368eeaa492996b170e P a45ecaf897aa437a99545955c113d3db: Bootstrap starting.
I20260807 18:15:44.337225 825 tablet_bootstrap.cc:492] T d8d8d275435d4b368eeaa492996b170e P 4fd10fc624034c9b9d25d1cbd38b9c4e: Bootstrap starting.
I20260807 18:15:44.337826 824 tablet_bootstrap.cc:654] T d8d8d275435d4b368eeaa492996b170e P a45ecaf897aa437a99545955c113d3db: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:44.339397 824 log.cc:824] T d8d8d275435d4b368eeaa492996b170e P a45ecaf897aa437a99545955c113d3db: Log is configured to *not* fsync() on all Append() calls
I20260807 18:15:44.339882 826 tablet_bootstrap.cc:492] T d8d8d275435d4b368eeaa492996b170e P 28556711425a498083dee80e1367e720: Bootstrap starting.
I20260807 18:15:44.341456 825 tablet_bootstrap.cc:654] T d8d8d275435d4b368eeaa492996b170e P 4fd10fc624034c9b9d25d1cbd38b9c4e: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:44.343118 825 log.cc:824] T d8d8d275435d4b368eeaa492996b170e P 4fd10fc624034c9b9d25d1cbd38b9c4e: Log is configured to *not* fsync() on all Append() calls
I20260807 18:15:44.344110 826 tablet_bootstrap.cc:654] T d8d8d275435d4b368eeaa492996b170e P 28556711425a498083dee80e1367e720: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:44.345500 826 log.cc:824] T d8d8d275435d4b368eeaa492996b170e P 28556711425a498083dee80e1367e720: Log is configured to *not* fsync() on all Append() calls
I20260807 18:15:44.346539 824 tablet_bootstrap.cc:492] T d8d8d275435d4b368eeaa492996b170e P a45ecaf897aa437a99545955c113d3db: No bootstrap required, opened a new log
I20260807 18:15:44.346923 824 ts_tablet_manager.cc:1397] T d8d8d275435d4b368eeaa492996b170e P a45ecaf897aa437a99545955c113d3db: Time spent bootstrapping tablet: real 0.013s user 0.011s sys 0.000s
I20260807 18:15:44.349485 825 tablet_bootstrap.cc:492] T d8d8d275435d4b368eeaa492996b170e P 4fd10fc624034c9b9d25d1cbd38b9c4e: No bootstrap required, opened a new log
I20260807 18:15:44.349484 826 tablet_bootstrap.cc:492] T d8d8d275435d4b368eeaa492996b170e P 28556711425a498083dee80e1367e720: No bootstrap required, opened a new log
I20260807 18:15:44.349835 826 ts_tablet_manager.cc:1397] T d8d8d275435d4b368eeaa492996b170e P 28556711425a498083dee80e1367e720: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 18:15:44.349836 825 ts_tablet_manager.cc:1397] T d8d8d275435d4b368eeaa492996b170e P 4fd10fc624034c9b9d25d1cbd38b9c4e: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.005s
I20260807 18:15:44.355608 824 raft_consensus.cc:348] T d8d8d275435d4b368eeaa492996b170e P a45ecaf897aa437a99545955c113d3db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd10fc624034c9b9d25d1cbd38b9c4e" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 38591 } } peers { permanent_uuid: "a45ecaf897aa437a99545955c113d3db" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 44477 } } peers { permanent_uuid: "28556711425a498083dee80e1367e720" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 45705 } }
I20260807 18:15:44.356205 824 raft_consensus.cc:374] T d8d8d275435d4b368eeaa492996b170e P a45ecaf897aa437a99545955c113d3db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:44.356317 824 raft_consensus.cc:729] T d8d8d275435d4b368eeaa492996b170e P a45ecaf897aa437a99545955c113d3db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45ecaf897aa437a99545955c113d3db, State: Initialized, Role: FOLLOWER
I20260807 18:15:44.357025 824 consensus_queue.cc:260] T d8d8d275435d4b368eeaa492996b170e P a45ecaf897aa437a99545955c113d3db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd10fc624034c9b9d25d1cbd38b9c4e" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 38591 } } peers { permanent_uuid: "a45ecaf897aa437a99545955c113d3db" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 44477 } } peers { permanent_uuid: "28556711425a498083dee80e1367e720" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 45705 } }
I20260807 18:15:44.359186 824 ts_tablet_manager.cc:1428] T d8d8d275435d4b368eeaa492996b170e P a45ecaf897aa437a99545955c113d3db: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260807 18:15:44.363086 825 raft_consensus.cc:348] T d8d8d275435d4b368eeaa492996b170e P 4fd10fc624034c9b9d25d1cbd38b9c4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd10fc624034c9b9d25d1cbd38b9c4e" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 38591 } } peers { permanent_uuid: "a45ecaf897aa437a99545955c113d3db" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 44477 } } peers { permanent_uuid: "28556711425a498083dee80e1367e720" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 45705 } }
I20260807 18:15:44.363081 826 raft_consensus.cc:348] T d8d8d275435d4b368eeaa492996b170e P 28556711425a498083dee80e1367e720 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd10fc624034c9b9d25d1cbd38b9c4e" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 38591 } } peers { permanent_uuid: "a45ecaf897aa437a99545955c113d3db" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 44477 } } peers { permanent_uuid: "28556711425a498083dee80e1367e720" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 45705 } }
I20260807 18:15:44.363669 825 raft_consensus.cc:374] T d8d8d275435d4b368eeaa492996b170e P 4fd10fc624034c9b9d25d1cbd38b9c4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:44.363669 826 raft_consensus.cc:374] T d8d8d275435d4b368eeaa492996b170e P 28556711425a498083dee80e1367e720 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:44.363802 825 raft_consensus.cc:729] T d8d8d275435d4b368eeaa492996b170e P 4fd10fc624034c9b9d25d1cbd38b9c4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fd10fc624034c9b9d25d1cbd38b9c4e, State: Initialized, Role: FOLLOWER
I20260807 18:15:44.363795 826 raft_consensus.cc:729] T d8d8d275435d4b368eeaa492996b170e P 28556711425a498083dee80e1367e720 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28556711425a498083dee80e1367e720, State: Initialized, Role: FOLLOWER
I20260807 18:15:44.364598 825 consensus_queue.cc:260] T d8d8d275435d4b368eeaa492996b170e P 4fd10fc624034c9b9d25d1cbd38b9c4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd10fc624034c9b9d25d1cbd38b9c4e" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 38591 } } peers { permanent_uuid: "a45ecaf897aa437a99545955c113d3db" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 44477 } } peers { permanent_uuid: "28556711425a498083dee80e1367e720" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 45705 } }
I20260807 18:15:44.364600 826 consensus_queue.cc:260] T d8d8d275435d4b368eeaa492996b170e P 28556711425a498083dee80e1367e720 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd10fc624034c9b9d25d1cbd38b9c4e" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 38591 } } peers { permanent_uuid: "a45ecaf897aa437a99545955c113d3db" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 44477 } } peers { permanent_uuid: "28556711425a498083dee80e1367e720" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 45705 } }
I20260807 18:15:44.365954 796 heartbeater.cc:503] Master 127.17.110.62:41333 was elected leader, sending a full tablet report...
I20260807 18:15:44.367259 825 ts_tablet_manager.cc:1428] T d8d8d275435d4b368eeaa492996b170e P 4fd10fc624034c9b9d25d1cbd38b9c4e: Time spent starting tablet: real 0.017s user 0.015s sys 0.004s
I20260807 18:15:44.368100 591 heartbeater.cc:503] Master 127.17.110.62:41333 was elected leader, sending a full tablet report...
I20260807 18:15:44.369282 826 ts_tablet_manager.cc:1428] T d8d8d275435d4b368eeaa492996b170e P 28556711425a498083dee80e1367e720: Time spent starting tablet: real 0.019s user 0.016s sys 0.005s
W20260807 18:15:44.394798 592 tablet.cc:2369] T d8d8d275435d4b368eeaa492996b170e P 28556711425a498083dee80e1367e720: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:15:44.447685 831 raft_consensus.cc:482] T d8d8d275435d4b368eeaa492996b170e P a45ecaf897aa437a99545955c113d3db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:15:44.448074 831 raft_consensus.cc:504] T d8d8d275435d4b368eeaa492996b170e P a45ecaf897aa437a99545955c113d3db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd10fc624034c9b9d25d1cbd38b9c4e" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 38591 } } peers { permanent_uuid: "a45ecaf897aa437a99545955c113d3db" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 44477 } } peers { permanent_uuid: "28556711425a498083dee80e1367e720" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 45705 } }
I20260807 18:15:44.450191 831 leader_election.cc:288] T d8d8d275435d4b368eeaa492996b170e P a45ecaf897aa437a99545955c113d3db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4fd10fc624034c9b9d25d1cbd38b9c4e (127.17.110.3:38591), 28556711425a498083dee80e1367e720 (127.17.110.2:45705)
I20260807 18:15:44.459697 817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:44.450780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:51397 (local address 127.17.110.3:38591)
0807 18:15:44.451009 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:44.451018 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:44.451626 (+ 608us) server_negotiation.cc:407] Connection header received
0807 18:15:44.451910 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:44.451941 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:44.452172 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:44.452287 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:15:44.453104 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.454410 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.457217 (+ 2807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.458408 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.458624 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:44.458761 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:44.459218 (+ 457us) server_negotiation.cc:299] Negotiation successful
0807 18:15:44.459369 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 18:15:44.459697 823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:44.450926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:56837 (local address 127.17.110.2:45705)
0807 18:15:44.451250 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:44.451259 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:44.452020 (+ 761us) server_negotiation.cc:407] Connection header received
0807 18:15:44.452260 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:44.452265 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:44.452362 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:44.452468 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:15:44.453229 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.454410 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.457152 (+ 2742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.458466 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.458685 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:44.458798 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:44.459211 (+ 413us) server_negotiation.cc:299] Negotiation successful
0807 18:15:44.459364 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 18:15:44.459898 836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:44.450882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:45705 (local address 127.17.110.1:56837)
0807 18:15:44.451858 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:44.451882 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:44.452095 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:44.452503 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:44.452508 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:44.452558 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:15:44.452984 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:44.452994 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.454664 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.454668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:44.456922 (+ 2254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:44.456944 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.458810 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.458817 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:44.459017 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:44.459053 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 18:15:44.459181 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 18:15:44.459364 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":862,"mutex_wait_us":58,"thread_start_us":570,"threads_started":1}
I20260807 18:15:44.459942 834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:44.450521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:38591 (local address 127.17.110.1:51397)
0807 18:15:44.451480 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:44.451504 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:44.451733 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:44.452310 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:44.452315 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:44.452344 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:15:44.452917 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:44.452931 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.454632 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.454638 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:44.457004 (+ 2366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:44.457019 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.458761 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.458771 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:44.459010 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:44.459048 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 18:15:44.459179 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 18:15:44.459364 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"mutex_wait_us":38,"thread_start_us":532,"threads_started":1}
I20260807 18:15:44.461030 722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8d8d275435d4b368eeaa492996b170e" candidate_uuid: "a45ecaf897aa437a99545955c113d3db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fd10fc624034c9b9d25d1cbd38b9c4e" is_pre_election: true
I20260807 18:15:44.461812 722 raft_consensus.cc:2454] T d8d8d275435d4b368eeaa492996b170e P 4fd10fc624034c9b9d25d1cbd38b9c4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a45ecaf897aa437a99545955c113d3db in term 0.
I20260807 18:15:44.461704 531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8d8d275435d4b368eeaa492996b170e" candidate_uuid: "a45ecaf897aa437a99545955c113d3db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28556711425a498083dee80e1367e720" is_pre_election: true
I20260807 18:15:44.462383 531 raft_consensus.cc:2454] T d8d8d275435d4b368eeaa492996b170e P 28556711425a498083dee80e1367e720 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a45ecaf897aa437a99545955c113d3db in term 0.
I20260807 18:15:44.462953 32706 leader_election.cc:302] T d8d8d275435d4b368eeaa492996b170e P a45ecaf897aa437a99545955c113d3db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fd10fc624034c9b9d25d1cbd38b9c4e, a45ecaf897aa437a99545955c113d3db; no voters:
I20260807 18:15:44.463558 831 raft_consensus.cc:2781] T d8d8d275435d4b368eeaa492996b170e P a45ecaf897aa437a99545955c113d3db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:15:44.463761 831 raft_consensus.cc:482] T d8d8d275435d4b368eeaa492996b170e P a45ecaf897aa437a99545955c113d3db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:15:44.463897 831 raft_consensus.cc:3037] T d8d8d275435d4b368eeaa492996b170e P a45ecaf897aa437a99545955c113d3db [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:44.467123 831 raft_consensus.cc:504] T d8d8d275435d4b368eeaa492996b170e P a45ecaf897aa437a99545955c113d3db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd10fc624034c9b9d25d1cbd38b9c4e" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 38591 } } peers { permanent_uuid: "a45ecaf897aa437a99545955c113d3db" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 44477 } } peers { permanent_uuid: "28556711425a498083dee80e1367e720" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 45705 } }
I20260807 18:15:44.467942 831 leader_election.cc:288] T d8d8d275435d4b368eeaa492996b170e P a45ecaf897aa437a99545955c113d3db [CANDIDATE]: Term 1 election: Requested vote from peers 4fd10fc624034c9b9d25d1cbd38b9c4e (127.17.110.3:38591), 28556711425a498083dee80e1367e720 (127.17.110.2:45705)
I20260807 18:15:44.468703 722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8d8d275435d4b368eeaa492996b170e" candidate_uuid: "a45ecaf897aa437a99545955c113d3db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fd10fc624034c9b9d25d1cbd38b9c4e"
I20260807 18:15:44.468703 531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8d8d275435d4b368eeaa492996b170e" candidate_uuid: "a45ecaf897aa437a99545955c113d3db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28556711425a498083dee80e1367e720"
I20260807 18:15:44.468976 722 raft_consensus.cc:3037] T d8d8d275435d4b368eeaa492996b170e P 4fd10fc624034c9b9d25d1cbd38b9c4e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:44.468971 531 raft_consensus.cc:3037] T d8d8d275435d4b368eeaa492996b170e P 28556711425a498083dee80e1367e720 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:44.472872 531 raft_consensus.cc:2454] T d8d8d275435d4b368eeaa492996b170e P 28556711425a498083dee80e1367e720 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45ecaf897aa437a99545955c113d3db in term 1.
I20260807 18:15:44.472872 722 raft_consensus.cc:2454] T d8d8d275435d4b368eeaa492996b170e P 4fd10fc624034c9b9d25d1cbd38b9c4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45ecaf897aa437a99545955c113d3db in term 1.
W20260807 18:15:44.473529 798 tablet.cc:2369] T d8d8d275435d4b368eeaa492996b170e P 4fd10fc624034c9b9d25d1cbd38b9c4e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:15:44.473655 32707 leader_election.cc:302] T d8d8d275435d4b368eeaa492996b170e P a45ecaf897aa437a99545955c113d3db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28556711425a498083dee80e1367e720, a45ecaf897aa437a99545955c113d3db; no voters:
I20260807 18:15:44.474084 831 raft_consensus.cc:2781] T d8d8d275435d4b368eeaa492996b170e P a45ecaf897aa437a99545955c113d3db [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:15:44.474661 831 raft_consensus.cc:686] T d8d8d275435d4b368eeaa492996b170e P a45ecaf897aa437a99545955c113d3db [term 1 LEADER]: Becoming Leader. State: Replica: a45ecaf897aa437a99545955c113d3db, State: Running, Role: LEADER
I20260807 18:15:44.475252 831 consensus_queue.cc:237] T d8d8d275435d4b368eeaa492996b170e P a45ecaf897aa437a99545955c113d3db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd10fc624034c9b9d25d1cbd38b9c4e" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 38591 } } peers { permanent_uuid: "a45ecaf897aa437a99545955c113d3db" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 44477 } } peers { permanent_uuid: "28556711425a498083dee80e1367e720" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 45705 } }
I20260807 18:15:44.480630 32584 catalog_manager.cc:5495] T d8d8d275435d4b368eeaa492996b170e P a45ecaf897aa437a99545955c113d3db reported cstate change: term changed from 0 to 1, leader changed from <none> to a45ecaf897aa437a99545955c113d3db (127.17.110.1). New cstate: current_term: 1 leader_uuid: "a45ecaf897aa437a99545955c113d3db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd10fc624034c9b9d25d1cbd38b9c4e" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 38591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45ecaf897aa437a99545955c113d3db" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 44477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28556711425a498083dee80e1367e720" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 45705 } health_report { overall_health: UNKNOWN } } }
I20260807 18:15:44.509902 818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:44.503928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42242 (local address 127.17.110.1:44477)
0807 18:15:44.504156 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:44.504162 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:44.504181 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:15:44.504374 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:44.504378 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:44.504480 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:44.504629 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:15:44.505007 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.505886 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.507454 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.507933 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.508057 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:44.508717 (+ 660us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:15:44.509345 (+ 628us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:15:44.509476 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:44.509601 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 18:15:44.509683 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 18:15:44.519798 531 raft_consensus.cc:1261] T d8d8d275435d4b368eeaa492996b170e P 28556711425a498083dee80e1367e720 [term 1 FOLLOWER]: Refusing update from remote peer a45ecaf897aa437a99545955c113d3db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:15:44.519789 722 raft_consensus.cc:1261] T d8d8d275435d4b368eeaa492996b170e P 4fd10fc624034c9b9d25d1cbd38b9c4e [term 1 FOLLOWER]: Refusing update from remote peer a45ecaf897aa437a99545955c113d3db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:15:44.520957 837 consensus_queue.cc:1035] T d8d8d275435d4b368eeaa492996b170e P a45ecaf897aa437a99545955c113d3db [LEADER]: Connected to new peer: Peer: permanent_uuid: "4fd10fc624034c9b9d25d1cbd38b9c4e" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 38591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:15:44.521377 831 consensus_queue.cc:1035] T d8d8d275435d4b368eeaa492996b170e P a45ecaf897aa437a99545955c113d3db [LEADER]: Connected to new peer: Peer: permanent_uuid: "28556711425a498083dee80e1367e720" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 45705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:15:44.535202 841 mvcc.cc:205] Tried to move back new op lower bound from 7315974326340571136 to 7315974326174408704. Current Snapshot: MvccSnapshot[applied={T|T < 7315974326340571136}]
I20260807 18:15:44.535454 842 mvcc.cc:205] Tried to move back new op lower bound from 7315974326340571136 to 7315974326174408704. Current Snapshot: MvccSnapshot[applied={T|T < 7315974326340571136}]
I20260807 18:15:44.549953 32584 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:15:44.550316 32584 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:15:44.555215 32584 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a63e7e43509844c194217b25cc2df981: Sending DeleteTablet for 3 replicas of tablet d8d8d275435d4b368eeaa492996b170e
I20260807 18:15:44.556946 32767 tablet_service.cc:1514] Processing DeleteTablet for tablet d8d8d275435d4b368eeaa492996b170e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:15:44 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:42236
I20260807 18:15:44.556968 503 tablet_service.cc:1514] Processing DeleteTablet for tablet d8d8d275435d4b368eeaa492996b170e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:15:44 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:35532
I20260807 18:15:44.557873 698 tablet_service.cc:1514] Processing DeleteTablet for tablet d8d8d275435d4b368eeaa492996b170e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:15:44 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:50604
I20260807 18:15:44.558254 855 tablet_replica.cc:330] T d8d8d275435d4b368eeaa492996b170e P a45ecaf897aa437a99545955c113d3db: stopping tablet replica
I20260807 18:15:44.558907 855 raft_consensus.cc:2229] T d8d8d275435d4b368eeaa492996b170e P a45ecaf897aa437a99545955c113d3db [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:15:44.559942 855 raft_consensus.cc:2258] T d8d8d275435d4b368eeaa492996b170e P a45ecaf897aa437a99545955c113d3db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:15:44.560621 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 32683
I20260807 18:15:44.563071 855 ts_tablet_manager.cc:1905] T d8d8d275435d4b368eeaa492996b170e P a45ecaf897aa437a99545955c113d3db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:15:44.568342 857 tablet_replica.cc:330] T d8d8d275435d4b368eeaa492996b170e P 4fd10fc624034c9b9d25d1cbd38b9c4e: stopping tablet replica
I20260807 18:15:44.568943 857 raft_consensus.cc:2229] T d8d8d275435d4b368eeaa492996b170e P 4fd10fc624034c9b9d25d1cbd38b9c4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:15:44.569579 857 raft_consensus.cc:2258] T d8d8d275435d4b368eeaa492996b170e P 4fd10fc624034c9b9d25d1cbd38b9c4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:15:44.570103 856 tablet_replica.cc:330] T d8d8d275435d4b368eeaa492996b170e P 28556711425a498083dee80e1367e720: stopping tablet replica
I20260807 18:15:44.570734 856 raft_consensus.cc:2229] T d8d8d275435d4b368eeaa492996b170e P 28556711425a498083dee80e1367e720 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:15:44.571455 856 raft_consensus.cc:2258] T d8d8d275435d4b368eeaa492996b170e P 28556711425a498083dee80e1367e720 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:15:44.573094 855 ts_tablet_manager.cc:1918] T d8d8d275435d4b368eeaa492996b170e P a45ecaf897aa437a99545955c113d3db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:15:44.573364 855 log.cc:1196] T d8d8d275435d4b368eeaa492996b170e P a45ecaf897aa437a99545955c113d3db: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-0/wal/wals/d8d8d275435d4b368eeaa492996b170e
I20260807 18:15:44.574095 855 ts_tablet_manager.cc:1939] T d8d8d275435d4b368eeaa492996b170e P a45ecaf897aa437a99545955c113d3db: Deleting consensus metadata
I20260807 18:15:44.574661 856 ts_tablet_manager.cc:1905] T d8d8d275435d4b368eeaa492996b170e P 28556711425a498083dee80e1367e720: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:15:44.576252 32568 catalog_manager.cc:4841] TS a45ecaf897aa437a99545955c113d3db (127.17.110.1:44477): tablet d8d8d275435d4b368eeaa492996b170e (table test-table [id=2cfba94ba36940d9aafbff9e1dd52341]) successfully deleted
I20260807 18:15:44.576634 857 ts_tablet_manager.cc:1905] T d8d8d275435d4b368eeaa492996b170e P 4fd10fc624034c9b9d25d1cbd38b9c4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:15:44.583010 856 ts_tablet_manager.cc:1918] T d8d8d275435d4b368eeaa492996b170e P 28556711425a498083dee80e1367e720: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:15:44.583247 856 log.cc:1196] T d8d8d275435d4b368eeaa492996b170e P 28556711425a498083dee80e1367e720: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-1/wal/wals/d8d8d275435d4b368eeaa492996b170e
I20260807 18:15:44.583887 856 ts_tablet_manager.cc:1939] T d8d8d275435d4b368eeaa492996b170e P 28556711425a498083dee80e1367e720: Deleting consensus metadata
I20260807 18:15:44.585047 857 ts_tablet_manager.cc:1918] T d8d8d275435d4b368eeaa492996b170e P 4fd10fc624034c9b9d25d1cbd38b9c4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:15:44.585268 857 log.cc:1196] T d8d8d275435d4b368eeaa492996b170e P 4fd10fc624034c9b9d25d1cbd38b9c4e: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126429568977-17848-0/minicluster-data/ts-2/wal/wals/d8d8d275435d4b368eeaa492996b170e
I20260807 18:15:44.585912 857 ts_tablet_manager.cc:1939] T d8d8d275435d4b368eeaa492996b170e P 4fd10fc624034c9b9d25d1cbd38b9c4e: Deleting consensus metadata
I20260807 18:15:44.586390 32568 catalog_manager.cc:4841] TS 28556711425a498083dee80e1367e720 (127.17.110.2:45705): tablet d8d8d275435d4b368eeaa492996b170e (table test-table [id=2cfba94ba36940d9aafbff9e1dd52341]) successfully deleted
I20260807 18:15:44.587906 32567 catalog_manager.cc:4841] TS 4fd10fc624034c9b9d25d1cbd38b9c4e (127.17.110.3:38591): tablet d8d8d275435d4b368eeaa492996b170e (table test-table [id=2cfba94ba36940d9aafbff9e1dd52341]) successfully deleted
Aug 07 18:15:44 dist-test-slave-39l8 krb5kdc[32525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.1@KRBTEST.COM
I20260807 18:15:44.604139 818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:44.562609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42246 (local address 127.17.110.1:44477)
0807 18:15:44.562894 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:44.562904 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:44.574761 (+ 11857us) server_negotiation.cc:407] Connection header received
0807 18:15:44.575150 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:44.575156 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:44.575268 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:44.575408 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:44.585028 (+ 9620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.586480 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.588785 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.589333 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.589458 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:44.595353 (+ 5895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:44.595376 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:44.595400 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:44.595472 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:44.600248 (+ 4776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:44.601199 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:44.601211 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:44.601215 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:44.601443 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:44.601974 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:44.601981 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:44.601985 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:44.602537 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:44.602961 (+ 424us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:44.603372 (+ 411us) server_negotiation.cc:299] Negotiation successful
0807 18:15:44.603682 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260807 18:15:44.605068 391 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:44.710800 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 32683
I20260807 18:15:44.738456 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 408
Aug 07 18:15:44 dist-test-slave-39l8 krb5kdc[32525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.2@KRBTEST.COM
I20260807 18:15:44.755024 823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:44.739389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35534 (local address 127.17.110.2:45705)
0807 18:15:44.739557 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:44.739562 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:44.739581 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:15:44.739734 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:44.739738 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:44.739833 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:44.739922 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:44.741203 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.741975 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.743504 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.743961 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.744067 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:44.748533 (+ 4466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:44.748551 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:44.748569 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:44.748612 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:44.751835 (+ 3223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:44.752801 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:44.752813 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:44.752817 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:44.753011 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:44.753459 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:44.753463 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:44.753465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:44.753902 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:44.754152 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:44.754421 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 18:15:44.754594 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 18:15:44.755805 586 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:44.849200 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 408
I20260807 18:15:44.880429 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 605
Aug 07 18:15:44 dist-test-slave-39l8 krb5kdc[32525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.3@KRBTEST.COM
I20260807 18:15:44.898674 817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:44.881699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50616 (local address 127.17.110.3:38591)
0807 18:15:44.881898 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:44.881903 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:44.881920 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:15:44.881976 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:44.881980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:44.882053 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:44.882140 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:44.883799 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.884558 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.886015 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.886493 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.886613 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:44.891740 (+ 5127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:44.891759 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:44.891777 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:44.891835 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:44.895233 (+ 3398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:44.896202 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:44.896212 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:44.896214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:44.896405 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:44.896855 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:44.896859 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:44.896862 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:44.897314 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:44.897574 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:44.898005 (+ 431us) server_negotiation.cc:299] Negotiation successful
0807 18:15:44.898294 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 18:15:44.899663 786 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:44.992393 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 605
I20260807 18:15:45.021394 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 32545
I20260807 18:15:45.022907 32640 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:45.137420 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 32545
2026-08-07T18:15:45Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4960 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:15:45 dist-test-slave-39l8 krb5kdc[868](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:15:45 dist-test-slave-39l8 krb5kdc[868](info): set up 2 sockets
Aug 07 18:15:45 dist-test-slave-39l8 krb5kdc[868](info): commencing operation
krb5kdc: starting...
W20260807 18:15:46.501062 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.306s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:15:46 dist-test-slave-39l8 krb5kdc[868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:15:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:15:46Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.17.110.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.17.110.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.17.110.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/rangeradmin_127.17.110.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.17.110.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/rangeradmin_127.17.110.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.17.110.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.17.110.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.17.110.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/rangerlookup_127.17.110.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.17.110.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/rangerlookup_127.17.110.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.110.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.17.110.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.110.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/HTTP_127.17.110.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.110.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/HTTP_127.17.110.20@KRBTEST.COM.keytab.
I20260807 18:15:46.942530 17848 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskDgKS9Z/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/postgres -l logfile start
2026-08-07 18:15:49.689 UTC [915] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 18:15:49.689 UTC [915] LOG: listening on IPv4 address "127.17.110.20", port 41345
2026-08-07 18:15:49.693 UTC [915] LOG: listening on Unix socket "/tmp/.s.PGSQL.41345"
2026-08-07 18:15:49.719 UTC [921] LOG: database system was shut down at 2026-08-07 18:15:48 UTC
2026-08-07 18:15:49.723 UTC [915] LOG: database system is ready to accept connections
I20260807 18:15:50.904516 17848 mini_postgres.cc:96] Postgres bound to 41345
2026-08-07 18:15:50.931 UTC [932] FATAL: database "slave" does not exist
127.17.110.20:41345 - accepting connections
I20260807 18:15:50.932149 17848 mini_ranger.cc:161] Starting Ranger...
I20260807 18:15:50.968875 17848 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 18:15:51.484211 17848 mini_ranger.cc:87] Created ranger Postgres database
I20260807 18:15:51.484411 17848 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ranger-admin
2026-08-07 18:15:51,904 [I] DB FLAVOR :POSTGRES
2026-08-07 18:15:51,904 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 18:15:51,904 [I] Checking connection..
2026-08-07 18:15:51,905 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:41345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 18:15:52,240 [I] Checking connection passed.
2026-08-07 18:15:52,241 [I] --------- Verifying version history table ---------
2026-08-07 18:15:52,241 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:41345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:15:52,533 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 18:15:52,534 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:41345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:15:52,807 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 18:15:52,807 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 18:15:52,807 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:41345/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 18:15:53.089 UTC [1015] WARNING: there is no transaction in progress
2026-08-07 18:15:53,099 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 18:15:53,099 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:41345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:15:53,358 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 18:15:53,358 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 18:15:53,359 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:41345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 18:15:53,626 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:41345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 18:15:53,896 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:41345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-39l8.c.gcp-upstream.internal','N') ;"
2026-08-07 18:15:54,176 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:41345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 18:15:54,481 [I] Table x_portal_user does not exist in database ranger
2026-08-07 18:15:54,482 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:41345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 18:15:54,769 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 18:15:54,769 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:41345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 18:15:55,049 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:41345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 18:15:55,341 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 18:15:55,341 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:41345/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 18:15:56.462 UTC [1177] WARNING: there is no transaction in progress
2026-08-07 18:15:56.481 UTC [1177] WARNING: there is no transaction in progress
2026-08-07 18:15:56.499 UTC [1177] WARNING: there is no transaction in progress
2026-08-07 18:15:56.518 UTC [1177] WARNING: there is no transaction in progress
2026-08-07 18:15:56.537 UTC [1177] WARNING: there is no transaction in progress
2026-08-07 18:15:56.559 UTC [1177] WARNING: there is no transaction in progress
2026-08-07 18:15:56.620 UTC [1177] WARNING: there is no transaction in progress
2026-08-07 18:15:56.628 UTC [1177] WARNING: there is no transaction in progress
2026-08-07 18:15:56.644 UTC [1177] WARNING: there is no transaction in progress
2026-08-07 18:15:56.656 UTC [1177] WARNING: there is no transaction in progress
2026-08-07 18:15:56.669 UTC [1177] WARNING: there is no transaction in progress
2026-08-07 18:15:56.685 UTC [1177] WARNING: there is no transaction in progress
2026-08-07 18:15:56.697 UTC [1177] WARNING: there is no transaction in progress
2026-08-07 18:15:56.704 UTC [1177] WARNING: there is no transaction in progress
2026-08-07 18:15:56.712 UTC [1177] WARNING: there is no transaction in progress
2026-08-07 18:15:56.719 UTC [1177] WARNING: there is no transaction in progress
2026-08-07 18:15:57,503 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 18:15:57,504 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:41345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 18:15:57,812 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 18:15:57,812 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:41345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 18:15:58,095 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 18:15:58,095 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:41345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 18:15:58,402 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 18:15:58,403 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:41345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 18:15:58,703 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:41345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 18:15:58,989 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:41345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-39l8.c.gcp-upstream.internal';"
2026-08-07 18:15:59,300 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 18:15:59,300 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:41345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 18:15:59,589 [I] DB_PATCHES have already been applied
I20260807 18:15:59.596863 17848 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ranger-admin:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskDgKS9Z/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 18:15:59.596977 17848 mini_ranger.cc:193] Using host: 127.17.110.20
I20260807 18:15:59.621779 17848 mini_ranger.cc:227] Ranger admin URL: http://127.17.110.20:36149
Aug 07, 2026 6:16:00 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:16:00 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:16:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 6:16:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 6:16:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 6:16:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 6:16:00 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 18:16:02.485816 17848 mini_ranger.cc:160] Time spent starting Ranger: real 11.554s user 0.005s sys 0.085s
Aug 07, 2026 6:16:24 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 6:16:24 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 07, 2026 6:16:24 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 6:16:24 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 6:16:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 6:16:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 6:16:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 6:16:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 6:16:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 6:16:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 6:16:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 6:16:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 6:16:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 6:16:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 6:16:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 6:16:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 6:16:24 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 6:16:25 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 18:16:26.523626 17848 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.17.110.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.17.110.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.17.110.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/rangerkms_127.17.110.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.17.110.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/rangerkms_127.17.110.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.110.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.17.110.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.110.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/HTTP_127.17.110.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.110.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/HTTP_127.17.110.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 18:16:26 dist-test-slave-39l8 krb5kdc[868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126586, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 18:16:26.758198 17848 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 18:16:26.793983 17848 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 18:16:27.631392 17848 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 18:16:27.631624 17848 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ranger-kms
I20260807 18:16:27.631771 17848 mini_ranger_kms.cc:227] Using postgres at 127.17.110.20:41345
2026-08-07 18:16:28,201 [I] DB FLAVOR :POSTGRES
2026-08-07 18:16:28,202 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 18:16:28,202 [I] Checking connection
2026-08-07 18:16:28,202 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:41345/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 18:16:28,514 [I] connection success
2026-08-07 18:16:28,514 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 18:16:28,514 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 18:16:28,515 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:41345/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 18:16:28,792 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 18:16:28,792 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 18:16:28,792 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 18:16:28,792 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:41345/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 18:16:29,127 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 18:16:29.697032 17848 mini_ranger_kms.cc:310] Created kms service
I20260807 18:16:29.908524 17848 mini_ranger_kms.cc:326] Created kudu user
I20260807 18:16:29.962019 17848 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 18:16:29 dist-test-slave-39l8 krb5kdc[868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126586, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.17.110.20@KRBTEST.COM
I20260807 18:16:30.441099 17848 mini_ranger_kms.cc:384] Added ranger policy
I20260807 18:16:30.442510 17848 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ranger-kms:/tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 18:16:30.442621 17848 mini_ranger_kms.cc:240] Using host: 127.17.110.20
I20260807 18:16:30.475697 17848 mini_ranger_kms.cc:276] Ranger KMS PID: 1451
I20260807 18:16:30.475919 17848 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.17.110.20:39011
Aug 07, 2026 6:16:31 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:16:31 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:16:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 6:16:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 6:16:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 6:16:32 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 18:16:32 dist-test-slave-39l8 krb5kdc[868](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786126592, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.110.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:16:33.362219 17848 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.604s user 0.144s sys 0.472s
I20260807 18:16:33.362628 17848 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 18:16:33.362911 17848 mini_ranger_kms.cc:401] 127.17.110.20:39011/kms/v1/keys
Aug 07 18:16:33 dist-test-slave-39l8 krb5kdc[868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126592, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.110.20@KRBTEST.COM for HTTP/127.17.110.20@KRBTEST.COM
Aug 07 18:16:33 dist-test-slave-39l8 krb5kdc[868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126592, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.110.20@KRBTEST.COM for HTTP/127.17.110.20@KRBTEST.COM
Aug 07 18:16:36 dist-test-slave-39l8 krb5kdc[868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.17.110.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
I20260807 18:16:36.532083 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:45411
--webserver_interface=127.17.110.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:44075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:45411
--ranger_config_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.110.20:39011/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:16:36.993239 1526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:16:36.993594 1526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:16:36.993700 1526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:16:37.001825 1526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:16:37.001965 1526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:16:37.002022 1526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:16:37.002085 1526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:16:37.002137 1526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:16:37.013811 1526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:44075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/master-0/wal
--ranger_kms_url=127.17.110.20:39011/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:45411
--ranger_config_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:45411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:16:37.015383 1526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:16:37.016791 1526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:16:37.032083 1533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:37.031980 1536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:37.031883 1534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:16:37.033794 1526 server_base.cc:1034] running on GCE node
I20260807 18:16:37.034914 1526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:16:37.036702 1526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:16:37.037968 1526 hybrid_clock.cc:648] HybridClock initialized: now 1786126597037911 us; error 67 us; skew 500 ppm
I20260807 18:16:37.038450 1526 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:16:37 dist-test-slave-39l8 krb5kdc[868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126597, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:16:37.046696 1526 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:16:37.048806 1526 webserver.cc:460] Webserver started at http://127.17.110.62:33669/ using document root <none> and password file <none>
I20260807 18:16:37.049659 1526 fs_manager.cc:359] Metadata directory not provided
I20260807 18:16:37.049889 1526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:16:37.050436 1526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:16:37 dist-test-slave-39l8 krb5kdc[868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126597, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for HTTP/127.17.110.20@KRBTEST.COM
I20260807 18:16:37.118772 1526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "1549e73d311f40d79f3fddb57bbc9ef1"
format_stamp: "Formatted at 2026-08-07 18:16:37 on dist-test-slave-39l8"
server_key: "e4f0faf0656d912210a0a7fc3d22e1da"
server_key_iv: "76d960d6a33e3cad956a4582813bdaa5"
server_key_version: "kuduclusterkey@0"
I20260807 18:16:37.120183 1526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "1549e73d311f40d79f3fddb57bbc9ef1"
format_stamp: "Formatted at 2026-08-07 18:16:37 on dist-test-slave-39l8"
server_key: "e4f0faf0656d912210a0a7fc3d22e1da"
server_key_iv: "76d960d6a33e3cad956a4582813bdaa5"
server_key_version: "kuduclusterkey@0"
I20260807 18:16:37.129034 1526 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.004s sys 0.004s
I20260807 18:16:37.150334 1548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:37.152274 1526 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 18:16:37.152549 1526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "1549e73d311f40d79f3fddb57bbc9ef1"
format_stamp: "Formatted at 2026-08-07 18:16:37 on dist-test-slave-39l8"
server_key: "e4f0faf0656d912210a0a7fc3d22e1da"
server_key_iv: "76d960d6a33e3cad956a4582813bdaa5"
server_key_version: "kuduclusterkey@0"
I20260807 18:16:37.152839 1526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:16:37.168443 1526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:16:37.174216 1526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:16:37.174672 1526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:16:37.203769 1526 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:45411
I20260807 18:16:37.203832 1617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:45411 every 8 connection(s)
I20260807 18:16:37.205694 1526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:16:37.206074 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 1526
I20260807 18:16:37.206323 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:16:37.207226 17848 external_mini_cluster.cc:1428] Setting key cedad0da4f47bb083a8a8dd61708cbf0
I20260807 18:16:37.213719 1618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 18:16:37 dist-test-slave-39l8 krb5kdc[868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:16:37.233479 1618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1549e73d311f40d79f3fddb57bbc9ef1: Bootstrap starting.
I20260807 18:16:37.238286 1618 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1549e73d311f40d79f3fddb57bbc9ef1: Neither blocks nor log segments found. Creating new log.
I20260807 18:16:37.240134 1618 log.cc:824] T 00000000000000000000000000000000 P 1549e73d311f40d79f3fddb57bbc9ef1: Log is configured to *not* fsync() on all Append() calls
I20260807 18:16:37.244612 1618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1549e73d311f40d79f3fddb57bbc9ef1: No bootstrap required, opened a new log
I20260807 18:16:37.244871 1623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:37.215010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52218 (local address 127.17.110.62:45411)
0807 18:16:37.216370 (+ 1360us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:37.216384 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:37.216440 (+ 56us) server_negotiation.cc:407] Connection header received
0807 18:16:37.218546 (+ 2106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:37.218582 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:37.219220 (+ 638us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:37.219792 (+ 572us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:37.221549 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:37.223722 (+ 2173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:37.226322 (+ 2600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:37.227440 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:37.227671 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:37.233876 (+ 6205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:37.233890 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:37.233911 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:37.233977 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:37.238858 (+ 4881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:37.240099 (+ 1241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:37.240108 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:37.240119 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:37.240318 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:37.241078 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:37.241089 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:37.241094 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:37.242135 (+ 1041us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:37.242563 (+ 428us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:37.243146 (+ 583us) server_negotiation.cc:299] Negotiation successful
0807 18:16:37.243623 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1005,"thread_start_us":655,"threads_started":1}
I20260807 18:16:37.255668 1618 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1549e73d311f40d79f3fddb57bbc9ef1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1549e73d311f40d79f3fddb57bbc9ef1" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45411 } }
I20260807 18:16:37.256321 1618 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1549e73d311f40d79f3fddb57bbc9ef1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:16:37.256474 1618 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1549e73d311f40d79f3fddb57bbc9ef1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1549e73d311f40d79f3fddb57bbc9ef1, State: Initialized, Role: FOLLOWER
I20260807 18:16:37.257475 1618 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1549e73d311f40d79f3fddb57bbc9ef1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1549e73d311f40d79f3fddb57bbc9ef1" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45411 } }
I20260807 18:16:37.257983 1618 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1549e73d311f40d79f3fddb57bbc9ef1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:16:37.258213 1618 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1549e73d311f40d79f3fddb57bbc9ef1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:16:37.258554 1618 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1549e73d311f40d79f3fddb57bbc9ef1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:16:37.261380 1618 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1549e73d311f40d79f3fddb57bbc9ef1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1549e73d311f40d79f3fddb57bbc9ef1" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45411 } }
I20260807 18:16:37.262094 1618 leader_election.cc:302] T 00000000000000000000000000000000 P 1549e73d311f40d79f3fddb57bbc9ef1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1549e73d311f40d79f3fddb57bbc9ef1; no voters:
I20260807 18:16:37.263173 1618 leader_election.cc:288] T 00000000000000000000000000000000 P 1549e73d311f40d79f3fddb57bbc9ef1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:16:37.263376 1625 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1549e73d311f40d79f3fddb57bbc9ef1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:16:37.264712 1625 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1549e73d311f40d79f3fddb57bbc9ef1 [term 1 LEADER]: Becoming Leader. State: Replica: 1549e73d311f40d79f3fddb57bbc9ef1, State: Running, Role: LEADER
I20260807 18:16:37.265446 1625 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1549e73d311f40d79f3fddb57bbc9ef1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1549e73d311f40d79f3fddb57bbc9ef1" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45411 } }
I20260807 18:16:37.266326 1618 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1549e73d311f40d79f3fddb57bbc9ef1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:16:37.270912 1627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1549e73d311f40d79f3fddb57bbc9ef1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1549e73d311f40d79f3fddb57bbc9ef1. Latest consensus state: current_term: 1 leader_uuid: "1549e73d311f40d79f3fddb57bbc9ef1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1549e73d311f40d79f3fddb57bbc9ef1" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45411 } } }
I20260807 18:16:37.271466 1627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1549e73d311f40d79f3fddb57bbc9ef1 [sys.catalog]: This master's current role is: LEADER
I20260807 18:16:37.272166 1626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1549e73d311f40d79f3fddb57bbc9ef1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1549e73d311f40d79f3fddb57bbc9ef1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1549e73d311f40d79f3fddb57bbc9ef1" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45411 } } }
I20260807 18:16:37.272526 1626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1549e73d311f40d79f3fddb57bbc9ef1 [sys.catalog]: This master's current role is: LEADER
I20260807 18:16:37.275182 1634 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:16:37.282243 1634 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:16:37.283447 1618 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 18:16:37.308331 1634 catalog_manager.cc:1347] Generated new cluster ID: b1ac5705276b47a6a88e4df685f24862
I20260807 18:16:37.308518 1634 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:16:37.329316 1634 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:16:37.331143 1634 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:16:37.341835 1634 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1549e73d311f40d79f3fddb57bbc9ef1: Generated new TSK 0
I20260807 18:16:37.342993 1634 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.110.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
I20260807 18:16:38.222968 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.110.20:39011/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.1:0
--local_ip_for_outbound_sockets=127.17.110.1
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:45411
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:44075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:16:38 dist-test-slave-39l8 krb5kdc[868](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786126598, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 18:16:38.725522 1684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:16:38.726060 1684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:16:38.726292 1684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:16:38.738888 1684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:16:38.739104 1684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:16:38.739326 1684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.1
I20260807 18:16:38.756748 1684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:44075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.17.110.20:39011/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:45411
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.1
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:16:38.758947 1684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:16:38.761076 1684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:16:38.784103 1700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:38.788419 1701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:38.790412 1704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:16:38.791172 1684 server_base.cc:1034] running on GCE node
I20260807 18:16:38.792052 1684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:16:38.799535 1684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:16:38.800958 1684 hybrid_clock.cc:648] HybridClock initialized: now 1786126598800859 us; error 119 us; skew 500 ppm
I20260807 18:16:38.801412 1684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:16:38 dist-test-slave-39l8 krb5kdc[868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126598, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:16:38.805405 1684 init.cc:377] Logged in from keytab as kudu/127.17.110.1@KRBTEST.COM (short username kudu)
I20260807 18:16:38.808408 1684 webserver.cc:460] Webserver started at http://127.17.110.1:40489/ using document root <none> and password file <none>
I20260807 18:16:38.809330 1684 fs_manager.cc:359] Metadata directory not provided
I20260807 18:16:38.809507 1684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:16:38.809876 1684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:16:38 dist-test-slave-39l8 krb5kdc[868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126598, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for HTTP/127.17.110.20@KRBTEST.COM
I20260807 18:16:38.828329 1684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-0/data/instance:
uuid: "2ef449db5d3a489eb711bf4e769b510a"
format_stamp: "Formatted at 2026-08-07 18:16:38 on dist-test-slave-39l8"
server_key: "d1902b70a9fd0b3f0e637859c96df67d"
server_key_iv: "ea1eeca81e6096742e559167a8104454"
server_key_version: "kuduclusterkey@0"
I20260807 18:16:38.830201 1684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance:
uuid: "2ef449db5d3a489eb711bf4e769b510a"
format_stamp: "Formatted at 2026-08-07 18:16:38 on dist-test-slave-39l8"
server_key: "d1902b70a9fd0b3f0e637859c96df67d"
server_key_iv: "ea1eeca81e6096742e559167a8104454"
server_key_version: "kuduclusterkey@0"
I20260807 18:16:38.845726 1684 fs_manager.cc:691] Time spent creating directory manager: real 0.015s user 0.002s sys 0.010s
I20260807 18:16:38.874394 1712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:38.876782 1684 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.004s sys 0.006s
I20260807 18:16:38.877138 1684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-0/wal
uuid: "2ef449db5d3a489eb711bf4e769b510a"
format_stamp: "Formatted at 2026-08-07 18:16:38 on dist-test-slave-39l8"
server_key: "d1902b70a9fd0b3f0e637859c96df67d"
server_key_iv: "ea1eeca81e6096742e559167a8104454"
server_key_version: "kuduclusterkey@0"
I20260807 18:16:38.877539 1684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:16:38.929162 1684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:16:38.933909 1684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:16:38.934353 1684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:16:38.935667 1684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:16:38.937346 1684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:16:38.937461 1684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:38.937618 1684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:16:38.937707 1684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:39.032927 1684 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.1:39173
I20260807 18:16:39.032948 1863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.1:39173 every 8 connection(s)
I20260807 18:16:39.034627 1684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
I20260807 18:16:39.040030 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 1684
I20260807 18:16:39.040277 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance
I20260807 18:16:39.041113 17848 external_mini_cluster.cc:1428] Setting key fbba015a83d7211524495273e347dc57
WARNING: no policy specified for kudu/127.17.110.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.2@KRBTEST.COM" created.
Aug 07 18:16:39 dist-test-slave-39l8 krb5kdc[868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126598, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:16:39.105268 1869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:39.038252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:45385 (local address 127.17.110.62:45411)
0807 18:16:39.059160 (+ 20908us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:39.059171 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:39.062733 (+ 3562us) server_negotiation.cc:407] Connection header received
0807 18:16:39.068547 (+ 5814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:39.068557 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:39.068693 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:39.068795 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:39.074839 (+ 6044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:39.076001 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:39.081386 (+ 5385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:39.082144 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:39.082286 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:39.091564 (+ 9278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:39.091602 (+ 38us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:39.091609 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:39.091679 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:39.098839 (+ 7160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:39.101764 (+ 2925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:39.101771 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:39.101775 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:39.101977 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:39.102677 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:39.102683 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:39.102687 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:39.103213 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:39.103369 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:39.104526 (+ 1157us) server_negotiation.cc:299] Negotiation successful
0807 18:16:39.104817 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20698,"thread_start_us":20525,"threads_started":1}
I20260807 18:16:39.108834 1871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:39.039115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45411 (local address 127.17.110.1:45385)
0807 18:16:39.061694 (+ 22579us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:39.061772 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:39.068053 (+ 6281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:39.072661 (+ 4608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:39.072676 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:39.073402 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:16:39.074441 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:39.074486 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:39.076378 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:39.076384 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:39.078563 (+ 2179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:39.078596 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:39.083355 (+ 4759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:39.083365 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:39.083766 (+ 401us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:39.087214 (+ 3448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:16:39.087259 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:16:39.091151 (+ 3892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:16:39.100739 (+ 9588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:16:39.100751 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:16:39.100774 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:16:39.101559 (+ 785us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:16:39.102173 (+ 614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:16:39.102177 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:16:39.102180 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:16:39.102522 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:16:39.103478 (+ 956us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:16:39.103487 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:16:39.103996 (+ 509us) client_negotiation.cc:769] Sending connection context
0807 18:16:39.105558 (+ 1562us) client_negotiation.cc:240] Negotiation successful
0807 18:16:39.106234 (+ 676us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22170,"thread_start_us":18309,"threads_started":1}
I20260807 18:16:39.111539 1864 heartbeater.cc:347] Connected to a master server at 127.17.110.62:45411
I20260807 18:16:39.111963 1864 heartbeater.cc:464] Registering TS with master...
I20260807 18:16:39.113003 1864 heartbeater.cc:511] Master 127.17.110.62:45411 requested a full tablet report, sending...
I20260807 18:16:39.116142 1570 ts_manager.cc:194] Registered new tserver with Master: 2ef449db5d3a489eb711bf4e769b510a (127.17.110.1:39173)
I20260807 18:16:39.118222 1570 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.1@KRBTEST.COM'} at 127.17.110.1:45385
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
I20260807 18:16:39.136356 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.110.20:39011/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.2:0
--local_ip_for_outbound_sockets=127.17.110.2
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:45411
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:44075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:16:39 dist-test-slave-39l8 krb5kdc[868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126598, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for HTTP/127.17.110.20@KRBTEST.COM
Aug 07 18:16:39 dist-test-slave-39l8 krb5kdc[868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126598, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for HTTP/127.17.110.20@KRBTEST.COM
W20260807 18:16:39.747200 1874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:16:39.747677 1874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:16:39.747895 1874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:16:39.756794 1874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:16:39.756976 1874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:16:39.757195 1874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.2
I20260807 18:16:39.770684 1874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:44075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.17.110.20:39011/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:45411
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.2
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:16:39.772696 1874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:16:39.774650 1874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:16:39.805341 1887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:39.805598 1889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:39.806034 1892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:16:39.806691 1874 server_base.cc:1034] running on GCE node
I20260807 18:16:39.807566 1874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:16:39.811110 1874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:16:39.812417 1874 hybrid_clock.cc:648] HybridClock initialized: now 1786126599812326 us; error 81 us; skew 500 ppm
I20260807 18:16:39.812918 1874 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:16:39 dist-test-slave-39l8 krb5kdc[868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126599, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:16:39.818048 1874 init.cc:377] Logged in from keytab as kudu/127.17.110.2@KRBTEST.COM (short username kudu)
I20260807 18:16:39.820259 1874 webserver.cc:460] Webserver started at http://127.17.110.2:39217/ using document root <none> and password file <none>
I20260807 18:16:39.821321 1874 fs_manager.cc:359] Metadata directory not provided
I20260807 18:16:39.821489 1874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:16:39.821902 1874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:16:39 dist-test-slave-39l8 krb5kdc[868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126599, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for HTTP/127.17.110.20@KRBTEST.COM
I20260807 18:16:39.842223 1874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-1/data/instance:
uuid: "12b2a2e327ef42e58d42622290b21bd2"
format_stamp: "Formatted at 2026-08-07 18:16:39 on dist-test-slave-39l8"
server_key: "5f2ce4cff80a0406aefe0914f9de5d4d"
server_key_iv: "6104906e696a9abfb4e44371e4c9b4b7"
server_key_version: "kuduclusterkey@0"
I20260807 18:16:39.843547 1874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance:
uuid: "12b2a2e327ef42e58d42622290b21bd2"
format_stamp: "Formatted at 2026-08-07 18:16:39 on dist-test-slave-39l8"
server_key: "5f2ce4cff80a0406aefe0914f9de5d4d"
server_key_iv: "6104906e696a9abfb4e44371e4c9b4b7"
server_key_version: "kuduclusterkey@0"
I20260807 18:16:39.851449 1874 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260807 18:16:39.869329 1909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:39.871614 1874 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 18:16:39.871861 1874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-1/wal
uuid: "12b2a2e327ef42e58d42622290b21bd2"
format_stamp: "Formatted at 2026-08-07 18:16:39 on dist-test-slave-39l8"
server_key: "5f2ce4cff80a0406aefe0914f9de5d4d"
server_key_iv: "6104906e696a9abfb4e44371e4c9b4b7"
server_key_version: "kuduclusterkey@0"
I20260807 18:16:39.872175 1874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:16:39.889652 1874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:16:39.895193 1874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:16:39.895623 1874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:16:39.896934 1874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:16:39.898952 1874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:16:39.899086 1874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:39.899251 1874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:16:39.899338 1874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:39.948223 1874 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.2:41407
I20260807 18:16:39.948326 2078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.2:41407 every 8 connection(s)
I20260807 18:16:39.950305 1874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
I20260807 18:16:39.951840 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 1874
I20260807 18:16:39.952062 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance
I20260807 18:16:39.952682 17848 external_mini_cluster.cc:1428] Setting key 7506cee5d2202e2c84d4233ed3f47767
Aug 07 18:16:39 dist-test-slave-39l8 krb5kdc[868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126599, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.110.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.3@KRBTEST.COM" created.
I20260807 18:16:39.999788 2081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:39.959736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:52633 (local address 127.17.110.62:45411)
0807 18:16:39.974336 (+ 14600us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:39.974341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:39.974365 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:16:39.976164 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:39.976170 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:39.976290 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:39.976426 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:39.979189 (+ 2763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:39.980378 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:39.983673 (+ 3295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:39.984631 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:39.984812 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:39.990299 (+ 5487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:39.990310 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:39.990316 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:39.990377 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:39.994682 (+ 4305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:39.996110 (+ 1428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:39.996116 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:39.996119 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:39.996292 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:39.997066 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:39.997071 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:39.997074 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:39.997529 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:39.997714 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:39.998913 (+ 1199us) server_negotiation.cc:299] Negotiation successful
0807 18:16:39.999254 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14447,"thread_start_us":14627,"threads_started":1}
I20260807 18:16:40.001216 2083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:39.960756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45411 (local address 127.17.110.2:52633)
0807 18:16:39.971821 (+ 11065us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:39.971951 (+ 130us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:39.975696 (+ 3745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:39.976820 (+ 1124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:39.976837 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:39.977724 (+ 887us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:16:39.978863 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:39.978903 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:39.980615 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:39.980626 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:39.983418 (+ 2792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:39.983460 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:39.984974 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:39.984991 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:39.985425 (+ 434us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:39.986414 (+ 989us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:16:39.986456 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:16:39.989888 (+ 3432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:16:39.994986 (+ 5098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:16:39.994999 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:16:39.995023 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:16:39.995815 (+ 792us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:16:39.996467 (+ 652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:16:39.996473 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:16:39.996477 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:16:39.996876 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:16:39.997802 (+ 926us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:16:39.997811 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:16:39.998338 (+ 527us) client_negotiation.cc:769] Sending connection context
0807 18:16:39.998689 (+ 351us) client_negotiation.cc:240] Negotiation successful
0807 18:16:39.999386 (+ 697us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10505,"thread_start_us":10123,"threads_started":1}
I20260807 18:16:40.003021 2079 heartbeater.cc:347] Connected to a master server at 127.17.110.62:45411
I20260807 18:16:40.003434 2079 heartbeater.cc:464] Registering TS with master...
I20260807 18:16:40.004623 2079 heartbeater.cc:511] Master 127.17.110.62:45411 requested a full tablet report, sending...
I20260807 18:16:40.006791 1570 ts_manager.cc:194] Registered new tserver with Master: 12b2a2e327ef42e58d42622290b21bd2 (127.17.110.2:41407)
I20260807 18:16:40.008162 1570 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.2@KRBTEST.COM'} at 127.17.110.2:52633
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
I20260807 18:16:40.039414 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.110.20:39011/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.3:0
--local_ip_for_outbound_sockets=127.17.110.3
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:45411
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:44075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:16:40.122552 1864 heartbeater.cc:503] Master 127.17.110.62:45411 was elected leader, sending a full tablet report...
W20260807 18:16:40.426138 2090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:16:40.426527 2090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:16:40.426771 2090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:16:40.435138 2090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:16:40.435425 2090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:16:40.435683 2090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.3
I20260807 18:16:40.446614 2090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:44075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.17.110.20:39011/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:45411
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.3
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:16:40.448124 2090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:16:40.449537 2090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:16:40.464253 2096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:40.464320 2100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:40.464493 2097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:16:40.465909 2090 server_base.cc:1034] running on GCE node
I20260807 18:16:40.466850 2090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:16:40.468053 2090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:16:40.469307 2090 hybrid_clock.cc:648] HybridClock initialized: now 1786126600469224 us; error 44 us; skew 500 ppm
I20260807 18:16:40.469677 2090 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:16:40 dist-test-slave-39l8 krb5kdc[868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126600, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:16:40.473378 2090 init.cc:377] Logged in from keytab as kudu/127.17.110.3@KRBTEST.COM (short username kudu)
I20260807 18:16:40.475313 2090 webserver.cc:460] Webserver started at http://127.17.110.3:46461/ using document root <none> and password file <none>
I20260807 18:16:40.476187 2090 fs_manager.cc:359] Metadata directory not provided
I20260807 18:16:40.476341 2090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:16:40.476789 2090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:16:40 dist-test-slave-39l8 krb5kdc[868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126600, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for HTTP/127.17.110.20@KRBTEST.COM
I20260807 18:16:40.494694 2090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-2/data/instance:
uuid: "e60c0179b8fa4cdbb22837c0414b5562"
format_stamp: "Formatted at 2026-08-07 18:16:40 on dist-test-slave-39l8"
server_key: "9021d7186f6b9121121f9213a6c0222a"
server_key_iv: "95b5be6398415a1bf70627ca8db65f99"
server_key_version: "kuduclusterkey@0"
I20260807 18:16:40.495899 2090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance:
uuid: "e60c0179b8fa4cdbb22837c0414b5562"
format_stamp: "Formatted at 2026-08-07 18:16:40 on dist-test-slave-39l8"
server_key: "9021d7186f6b9121121f9213a6c0222a"
server_key_iv: "95b5be6398415a1bf70627ca8db65f99"
server_key_version: "kuduclusterkey@0"
I20260807 18:16:40.503124 2090 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260807 18:16:40.522512 2108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:40.524307 2090 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 18:16:40.524489 2090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-2/wal
uuid: "e60c0179b8fa4cdbb22837c0414b5562"
format_stamp: "Formatted at 2026-08-07 18:16:40 on dist-test-slave-39l8"
server_key: "9021d7186f6b9121121f9213a6c0222a"
server_key_iv: "95b5be6398415a1bf70627ca8db65f99"
server_key_version: "kuduclusterkey@0"
I20260807 18:16:40.524737 2090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:16:40.543190 2090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:16:40.547870 2090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:16:40.548177 2090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:16:40.549198 2090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:16:40.551052 2090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:16:40.551141 2090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:40.551240 2090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:16:40.551297 2090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:40.593832 2090 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.3:39265
I20260807 18:16:40.593915 2287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.3:39265 every 8 connection(s)
I20260807 18:16:40.595548 2090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
I20260807 18:16:40.605690 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 2090
I20260807 18:16:40.605893 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance
I20260807 18:16:40.606544 17848 external_mini_cluster.cc:1428] Setting key ba0bfd324541bb0b3835b8398cea0800
Aug 07 18:16:40 dist-test-slave-39l8 krb5kdc[868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126600, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:16:40.620049 2292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:40.598613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:48211 (local address 127.17.110.62:45411)
0807 18:16:40.599276 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:40.599281 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:40.600380 (+ 1099us) server_negotiation.cc:407] Connection header received
0807 18:16:40.602383 (+ 2003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:40.602389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:40.602498 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:40.602633 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:40.604271 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:40.605038 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:40.607558 (+ 2520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:40.608221 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:40.608380 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:40.612980 (+ 4600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:40.612985 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:40.612989 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:40.613030 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:40.616356 (+ 3326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:40.617284 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:40.617288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:40.617291 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:40.617438 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:40.617957 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:40.617960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:40.617963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:40.618260 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:40.618446 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:40.619368 (+ 922us) server_negotiation.cc:299] Negotiation successful
0807 18:16:40.619640 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":355,"threads_started":1}
I20260807 18:16:40.621119 2304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:40.599318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45411 (local address 127.17.110.3:48211)
0807 18:16:40.600182 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:40.600238 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:40.602056 (+ 1818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:40.602932 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:40.602941 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:40.603429 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:16:40.604069 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:40.604089 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:40.605235 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:40.605242 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:40.607363 (+ 2121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:40.607386 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:40.608494 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:40.608501 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:40.608890 (+ 389us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:40.609817 (+ 927us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:16:40.609843 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:16:40.612676 (+ 2833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:16:40.616578 (+ 3902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:16:40.616593 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:16:40.616610 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:16:40.617126 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:16:40.617571 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:16:40.617576 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:16:40.617580 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:16:40.617818 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:16:40.618419 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:16:40.618428 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:16:40.618945 (+ 517us) client_negotiation.cc:769] Sending connection context
0807 18:16:40.619253 (+ 308us) client_negotiation.cc:240] Negotiation successful
0807 18:16:40.619860 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":365,"threads_started":1}
I20260807 18:16:40.623050 2288 heartbeater.cc:347] Connected to a master server at 127.17.110.62:45411
I20260807 18:16:40.623457 2288 heartbeater.cc:464] Registering TS with master...
I20260807 18:16:40.624650 2288 heartbeater.cc:511] Master 127.17.110.62:45411 requested a full tablet report, sending...
I20260807 18:16:40.626473 1570 ts_manager.cc:194] Registered new tserver with Master: e60c0179b8fa4cdbb22837c0414b5562 (127.17.110.3:39265)
I20260807 18:16:40.627528 1570 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.3@KRBTEST.COM'} at 127.17.110.3:48211
I20260807 18:16:40.636574 17848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:16:40.655755 2292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:40.641793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52224 (local address 127.17.110.62:45411)
0807 18:16:40.642043 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:40.642051 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:40.642331 (+ 280us) server_negotiation.cc:407] Connection header received
0807 18:16:40.642556 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:40.642562 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:40.642654 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:40.642822 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:40.644482 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:40.645233 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:40.646657 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:40.647171 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:40.647318 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:40.649886 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:40.649891 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:40.649895 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:40.649934 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:40.652809 (+ 2875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:40.653608 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:40.653614 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:40.653617 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:40.653765 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:40.654171 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:40.654178 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:40.654181 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:40.654491 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:40.654814 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:40.655144 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 18:16:40.655409 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 18:16:40.660894 1570 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52224:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:16:40.669127 1570 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:16:40.698562 2319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:40.683688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:39173 (local address 127.0.0.1:43930)
0807 18:16:40.684904 (+ 1216us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:40.684960 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:40.685210 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:40.687777 (+ 2567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:40.687788 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:40.687834 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:16:40.688475 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:40.688493 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:40.691348 (+ 2855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:40.691358 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:40.694592 (+ 3234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:40.694643 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:40.697374 (+ 2731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:40.697382 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:40.697638 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:40.697692 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 18:16:40.697846 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 18:16:40.698076 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":838,"thread_start_us":8548,"threads_started":1}
I20260807 18:16:40.699293 2320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:40.684257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43930 (local address 127.17.110.1:39173)
0807 18:16:40.686523 (+ 2266us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:40.686536 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:40.686592 (+ 56us) server_negotiation.cc:407] Connection header received
0807 18:16:40.686761 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:40.686773 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:40.687111 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:40.687367 (+ 256us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:16:40.688779 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:40.691058 (+ 2279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:40.694910 (+ 3852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:40.696806 (+ 1896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:40.696952 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:40.697191 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:40.698767 (+ 1576us) server_negotiation.cc:299] Negotiation successful
0807 18:16:40.698954 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2063,"thread_start_us":1896,"threads_started":1}
I20260807 18:16:40.704053 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:40.684296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55930 (local address 127.17.110.2:41407)
0807 18:16:40.693647 (+ 9351us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:40.693657 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:40.693686 (+ 29us) server_negotiation.cc:407] Connection header received
0807 18:16:40.693801 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:40.693811 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:40.694101 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:40.696151 (+ 2050us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:16:40.696384 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:40.697884 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:40.701231 (+ 3347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:40.702839 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:40.703004 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:40.703233 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:40.703591 (+ 358us) server_negotiation.cc:299] Negotiation successful
0807 18:16:40.703766 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9163,"thread_start_us":9047,"threads_started":1}
I20260807 18:16:40.704025 2316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:40.683995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:41407 (local address 127.0.0.1:55930)
0807 18:16:40.692594 (+ 8599us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:40.692629 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:40.692805 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:40.694374 (+ 1569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:40.694383 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:40.694425 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:16:40.695005 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:40.695016 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:40.698064 (+ 3048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:40.698069 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:40.700973 (+ 2904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:40.700988 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:40.703073 (+ 2085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:40.703079 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:40.703368 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:40.703394 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:16:40.703500 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 18:16:40.703653 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8436,"thread_start_us":345,"threads_started":1}
I20260807 18:16:40.704026 1775 tablet_service.cc:1467] Processing CreateTablet for tablet 941d47956d8f4eeba3646926a95ed1ba (DEFAULT_TABLE table=test-table [id=aa3d09cef342442885d8fdb00da088d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:16:40.705088 2326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:40.686642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:39265 (local address 127.0.0.1:42714)
0807 18:16:40.692955 (+ 6313us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:40.692974 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:40.693125 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:40.697223 (+ 4098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:40.697227 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:40.697271 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:16:40.697757 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:40.697771 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:40.699825 (+ 2054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:40.699831 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:40.701860 (+ 2029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:40.701879 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:40.704235 (+ 2356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:40.704243 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:40.704423 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:40.704455 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 18:16:40.704551 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 18:16:40.704699 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6176,"thread_start_us":6291,"threads_started":1}
I20260807 18:16:40.705283 2327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:40.686575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42714 (local address 127.17.110.3:39265)
0807 18:16:40.695143 (+ 8568us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:40.695154 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:40.695207 (+ 53us) server_negotiation.cc:407] Connection header received
0807 18:16:40.695315 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:40.695324 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:40.695619 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:40.695850 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:16:40.698041 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:40.699648 (+ 1607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:40.702063 (+ 2415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:40.703642 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:40.703794 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:40.704041 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:40.704690 (+ 649us) server_negotiation.cc:299] Negotiation successful
0807 18:16:40.704908 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8285,"thread_start_us":8097,"threads_started":1}
I20260807 18:16:40.706094 1775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 941d47956d8f4eeba3646926a95ed1ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:16:40.708285 1973 tablet_service.cc:1467] Processing CreateTablet for tablet 941d47956d8f4eeba3646926a95ed1ba (DEFAULT_TABLE table=test-table [id=aa3d09cef342442885d8fdb00da088d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:16:40.709689 1973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 941d47956d8f4eeba3646926a95ed1ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:16:40.709223 2185 tablet_service.cc:1467] Processing CreateTablet for tablet 941d47956d8f4eeba3646926a95ed1ba (DEFAULT_TABLE table=test-table [id=aa3d09cef342442885d8fdb00da088d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:16:40.710809 2185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 941d47956d8f4eeba3646926a95ed1ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:16:40.721467 2328 tablet_bootstrap.cc:492] T 941d47956d8f4eeba3646926a95ed1ba P 12b2a2e327ef42e58d42622290b21bd2: Bootstrap starting.
I20260807 18:16:40.723908 2330 tablet_bootstrap.cc:492] T 941d47956d8f4eeba3646926a95ed1ba P 2ef449db5d3a489eb711bf4e769b510a: Bootstrap starting.
I20260807 18:16:40.726778 2328 tablet_bootstrap.cc:654] T 941d47956d8f4eeba3646926a95ed1ba P 12b2a2e327ef42e58d42622290b21bd2: Neither blocks nor log segments found. Creating new log.
I20260807 18:16:40.727392 2332 tablet_bootstrap.cc:492] T 941d47956d8f4eeba3646926a95ed1ba P e60c0179b8fa4cdbb22837c0414b5562: Bootstrap starting.
I20260807 18:16:40.728277 2330 tablet_bootstrap.cc:654] T 941d47956d8f4eeba3646926a95ed1ba P 2ef449db5d3a489eb711bf4e769b510a: Neither blocks nor log segments found. Creating new log.
I20260807 18:16:40.728406 2328 log.cc:824] T 941d47956d8f4eeba3646926a95ed1ba P 12b2a2e327ef42e58d42622290b21bd2: Log is configured to *not* fsync() on all Append() calls
I20260807 18:16:40.729830 2330 log.cc:824] T 941d47956d8f4eeba3646926a95ed1ba P 2ef449db5d3a489eb711bf4e769b510a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:16:40.731343 2332 tablet_bootstrap.cc:654] T 941d47956d8f4eeba3646926a95ed1ba P e60c0179b8fa4cdbb22837c0414b5562: Neither blocks nor log segments found. Creating new log.
I20260807 18:16:40.732426 2332 log.cc:824] T 941d47956d8f4eeba3646926a95ed1ba P e60c0179b8fa4cdbb22837c0414b5562: Log is configured to *not* fsync() on all Append() calls
I20260807 18:16:40.733067 2328 tablet_bootstrap.cc:492] T 941d47956d8f4eeba3646926a95ed1ba P 12b2a2e327ef42e58d42622290b21bd2: No bootstrap required, opened a new log
I20260807 18:16:40.733526 2328 ts_tablet_manager.cc:1397] T 941d47956d8f4eeba3646926a95ed1ba P 12b2a2e327ef42e58d42622290b21bd2: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260807 18:16:40.735759 2332 tablet_bootstrap.cc:492] T 941d47956d8f4eeba3646926a95ed1ba P e60c0179b8fa4cdbb22837c0414b5562: No bootstrap required, opened a new log
I20260807 18:16:40.736169 2332 ts_tablet_manager.cc:1397] T 941d47956d8f4eeba3646926a95ed1ba P e60c0179b8fa4cdbb22837c0414b5562: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260807 18:16:40.739817 2330 tablet_bootstrap.cc:492] T 941d47956d8f4eeba3646926a95ed1ba P 2ef449db5d3a489eb711bf4e769b510a: No bootstrap required, opened a new log
I20260807 18:16:40.740163 2330 ts_tablet_manager.cc:1397] T 941d47956d8f4eeba3646926a95ed1ba P 2ef449db5d3a489eb711bf4e769b510a: Time spent bootstrapping tablet: real 0.017s user 0.014s sys 0.000s
I20260807 18:16:40.744657 2332 raft_consensus.cc:348] T 941d47956d8f4eeba3646926a95ed1ba P e60c0179b8fa4cdbb22837c0414b5562 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef449db5d3a489eb711bf4e769b510a" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 39173 } } peers { permanent_uuid: "12b2a2e327ef42e58d42622290b21bd2" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 41407 } } peers { permanent_uuid: "e60c0179b8fa4cdbb22837c0414b5562" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 39265 } }
I20260807 18:16:40.745124 2332 raft_consensus.cc:374] T 941d47956d8f4eeba3646926a95ed1ba P e60c0179b8fa4cdbb22837c0414b5562 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:16:40.745234 2332 raft_consensus.cc:729] T 941d47956d8f4eeba3646926a95ed1ba P e60c0179b8fa4cdbb22837c0414b5562 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e60c0179b8fa4cdbb22837c0414b5562, State: Initialized, Role: FOLLOWER
I20260807 18:16:40.745077 2328 raft_consensus.cc:348] T 941d47956d8f4eeba3646926a95ed1ba P 12b2a2e327ef42e58d42622290b21bd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef449db5d3a489eb711bf4e769b510a" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 39173 } } peers { permanent_uuid: "12b2a2e327ef42e58d42622290b21bd2" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 41407 } } peers { permanent_uuid: "e60c0179b8fa4cdbb22837c0414b5562" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 39265 } }
I20260807 18:16:40.745525 2328 raft_consensus.cc:374] T 941d47956d8f4eeba3646926a95ed1ba P 12b2a2e327ef42e58d42622290b21bd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:16:40.745654 2328 raft_consensus.cc:729] T 941d47956d8f4eeba3646926a95ed1ba P 12b2a2e327ef42e58d42622290b21bd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12b2a2e327ef42e58d42622290b21bd2, State: Initialized, Role: FOLLOWER
I20260807 18:16:40.746097 2332 consensus_queue.cc:260] T 941d47956d8f4eeba3646926a95ed1ba P e60c0179b8fa4cdbb22837c0414b5562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef449db5d3a489eb711bf4e769b510a" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 39173 } } peers { permanent_uuid: "12b2a2e327ef42e58d42622290b21bd2" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 41407 } } peers { permanent_uuid: "e60c0179b8fa4cdbb22837c0414b5562" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 39265 } }
I20260807 18:16:40.746265 2328 consensus_queue.cc:260] T 941d47956d8f4eeba3646926a95ed1ba P 12b2a2e327ef42e58d42622290b21bd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef449db5d3a489eb711bf4e769b510a" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 39173 } } peers { permanent_uuid: "12b2a2e327ef42e58d42622290b21bd2" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 41407 } } peers { permanent_uuid: "e60c0179b8fa4cdbb22837c0414b5562" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 39265 } }
I20260807 18:16:40.747605 2288 heartbeater.cc:503] Master 127.17.110.62:45411 was elected leader, sending a full tablet report...
I20260807 18:16:40.748270 2332 ts_tablet_manager.cc:1428] T 941d47956d8f4eeba3646926a95ed1ba P e60c0179b8fa4cdbb22837c0414b5562: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260807 18:16:40.752429 2079 heartbeater.cc:503] Master 127.17.110.62:45411 was elected leader, sending a full tablet report...
I20260807 18:16:40.752779 2330 raft_consensus.cc:348] T 941d47956d8f4eeba3646926a95ed1ba P 2ef449db5d3a489eb711bf4e769b510a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef449db5d3a489eb711bf4e769b510a" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 39173 } } peers { permanent_uuid: "12b2a2e327ef42e58d42622290b21bd2" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 41407 } } peers { permanent_uuid: "e60c0179b8fa4cdbb22837c0414b5562" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 39265 } }
I20260807 18:16:40.753340 2330 raft_consensus.cc:374] T 941d47956d8f4eeba3646926a95ed1ba P 2ef449db5d3a489eb711bf4e769b510a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:16:40.752432 2328 ts_tablet_manager.cc:1428] T 941d47956d8f4eeba3646926a95ed1ba P 12b2a2e327ef42e58d42622290b21bd2: Time spent starting tablet: real 0.019s user 0.014s sys 0.003s
I20260807 18:16:40.753463 2330 raft_consensus.cc:729] T 941d47956d8f4eeba3646926a95ed1ba P 2ef449db5d3a489eb711bf4e769b510a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ef449db5d3a489eb711bf4e769b510a, State: Initialized, Role: FOLLOWER
I20260807 18:16:40.754264 2330 consensus_queue.cc:260] T 941d47956d8f4eeba3646926a95ed1ba P 2ef449db5d3a489eb711bf4e769b510a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef449db5d3a489eb711bf4e769b510a" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 39173 } } peers { permanent_uuid: "12b2a2e327ef42e58d42622290b21bd2" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 41407 } } peers { permanent_uuid: "e60c0179b8fa4cdbb22837c0414b5562" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 39265 } }
I20260807 18:16:40.756294 2330 ts_tablet_manager.cc:1428] T 941d47956d8f4eeba3646926a95ed1ba P 2ef449db5d3a489eb711bf4e769b510a: Time spent starting tablet: real 0.016s user 0.011s sys 0.005s
I20260807 18:16:40.820951 2337 raft_consensus.cc:482] T 941d47956d8f4eeba3646926a95ed1ba P 12b2a2e327ef42e58d42622290b21bd2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:16:40.821295 2337 raft_consensus.cc:504] T 941d47956d8f4eeba3646926a95ed1ba P 12b2a2e327ef42e58d42622290b21bd2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef449db5d3a489eb711bf4e769b510a" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 39173 } } peers { permanent_uuid: "12b2a2e327ef42e58d42622290b21bd2" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 41407 } } peers { permanent_uuid: "e60c0179b8fa4cdbb22837c0414b5562" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 39265 } }
W20260807 18:16:40.822906 1866 tablet.cc:2369] T 941d47956d8f4eeba3646926a95ed1ba P 2ef449db5d3a489eb711bf4e769b510a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:16:40.823194 2337 leader_election.cc:288] T 941d47956d8f4eeba3646926a95ed1ba P 12b2a2e327ef42e58d42622290b21bd2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ef449db5d3a489eb711bf4e769b510a (127.17.110.1:39173), e60c0179b8fa4cdbb22837c0414b5562 (127.17.110.3:39265)
I20260807 18:16:40.832911 2327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:40.824244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:57159 (local address 127.17.110.3:39265)
0807 18:16:40.824505 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:40.824514 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:40.824695 (+ 181us) server_negotiation.cc:407] Connection header received
0807 18:16:40.824968 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:40.824974 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:40.825060 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:40.825170 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:16:40.825837 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:40.827124 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:40.830236 (+ 3112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:40.831611 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:40.831809 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:40.831889 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:40.832480 (+ 591us) server_negotiation.cc:299] Negotiation successful
0807 18:16:40.832635 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 18:16:40.832913 2320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:40.823846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:53293 (local address 127.17.110.1:39173)
0807 18:16:40.824243 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:40.824250 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:40.825024 (+ 774us) server_negotiation.cc:407] Connection header received
0807 18:16:40.825238 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:40.825245 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:40.825326 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:40.825407 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:16:40.826072 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:40.827181 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:40.830237 (+ 3056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:40.831652 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:40.831809 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:40.831889 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:40.832480 (+ 591us) server_negotiation.cc:299] Negotiation successful
0807 18:16:40.832652 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216}
I20260807 18:16:40.833112 2341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:40.823676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:39173 (local address 127.17.110.2:53293)
0807 18:16:40.824912 (+ 1236us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:40.824929 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:40.825073 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:40.825458 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:40.825465 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:40.825491 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:16:40.825896 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:40.825905 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:40.827306 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:40.827310 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:40.830024 (+ 2714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:40.830042 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:40.831930 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:40.831938 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:40.832277 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:40.832305 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 18:16:40.832406 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 18:16:40.832592 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1098,"mutex_wait_us":238,"thread_start_us":878,"threads_started":1}
I20260807 18:16:40.833384 2340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:40.823515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:39265 (local address 127.17.110.2:57159)
0807 18:16:40.824547 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:40.824573 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:40.824793 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:40.825184 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:40.825191 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:40.825221 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:16:40.825657 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:40.825675 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:40.827295 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:40.827302 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:40.830026 (+ 2724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:40.830042 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:40.831935 (+ 1893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:40.831940 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:40.832255 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:40.832292 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 18:16:40.832406 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 18:16:40.832592 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":819,"mutex_wait_us":230,"thread_start_us":616,"threads_started":1}
I20260807 18:16:40.834363 1798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "941d47956d8f4eeba3646926a95ed1ba" candidate_uuid: "12b2a2e327ef42e58d42622290b21bd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ef449db5d3a489eb711bf4e769b510a" is_pre_election: true
I20260807 18:16:40.834628 2217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "941d47956d8f4eeba3646926a95ed1ba" candidate_uuid: "12b2a2e327ef42e58d42622290b21bd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e60c0179b8fa4cdbb22837c0414b5562" is_pre_election: true
I20260807 18:16:40.834956 1798 raft_consensus.cc:2454] T 941d47956d8f4eeba3646926a95ed1ba P 2ef449db5d3a489eb711bf4e769b510a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12b2a2e327ef42e58d42622290b21bd2 in term 0.
I20260807 18:16:40.835323 2217 raft_consensus.cc:2454] T 941d47956d8f4eeba3646926a95ed1ba P e60c0179b8fa4cdbb22837c0414b5562 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12b2a2e327ef42e58d42622290b21bd2 in term 0.
I20260807 18:16:40.835747 1910 leader_election.cc:302] T 941d47956d8f4eeba3646926a95ed1ba P 12b2a2e327ef42e58d42622290b21bd2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12b2a2e327ef42e58d42622290b21bd2, 2ef449db5d3a489eb711bf4e769b510a; no voters:
I20260807 18:16:40.836241 2337 raft_consensus.cc:2781] T 941d47956d8f4eeba3646926a95ed1ba P 12b2a2e327ef42e58d42622290b21bd2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:16:40.836409 2337 raft_consensus.cc:482] T 941d47956d8f4eeba3646926a95ed1ba P 12b2a2e327ef42e58d42622290b21bd2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:16:40.836530 2337 raft_consensus.cc:3037] T 941d47956d8f4eeba3646926a95ed1ba P 12b2a2e327ef42e58d42622290b21bd2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:16:40.839241 2337 raft_consensus.cc:504] T 941d47956d8f4eeba3646926a95ed1ba P 12b2a2e327ef42e58d42622290b21bd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef449db5d3a489eb711bf4e769b510a" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 39173 } } peers { permanent_uuid: "12b2a2e327ef42e58d42622290b21bd2" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 41407 } } peers { permanent_uuid: "e60c0179b8fa4cdbb22837c0414b5562" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 39265 } }
I20260807 18:16:40.840059 2337 leader_election.cc:288] T 941d47956d8f4eeba3646926a95ed1ba P 12b2a2e327ef42e58d42622290b21bd2 [CANDIDATE]: Term 1 election: Requested vote from peers 2ef449db5d3a489eb711bf4e769b510a (127.17.110.1:39173), e60c0179b8fa4cdbb22837c0414b5562 (127.17.110.3:39265)
I20260807 18:16:40.840710 1798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "941d47956d8f4eeba3646926a95ed1ba" candidate_uuid: "12b2a2e327ef42e58d42622290b21bd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ef449db5d3a489eb711bf4e769b510a"
I20260807 18:16:40.840968 1798 raft_consensus.cc:3037] T 941d47956d8f4eeba3646926a95ed1ba P 2ef449db5d3a489eb711bf4e769b510a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:16:40.841112 2217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "941d47956d8f4eeba3646926a95ed1ba" candidate_uuid: "12b2a2e327ef42e58d42622290b21bd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e60c0179b8fa4cdbb22837c0414b5562"
I20260807 18:16:40.841364 2217 raft_consensus.cc:3037] T 941d47956d8f4eeba3646926a95ed1ba P e60c0179b8fa4cdbb22837c0414b5562 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:16:40.843714 1798 raft_consensus.cc:2454] T 941d47956d8f4eeba3646926a95ed1ba P 2ef449db5d3a489eb711bf4e769b510a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12b2a2e327ef42e58d42622290b21bd2 in term 1.
I20260807 18:16:40.843845 2217 raft_consensus.cc:2454] T 941d47956d8f4eeba3646926a95ed1ba P e60c0179b8fa4cdbb22837c0414b5562 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12b2a2e327ef42e58d42622290b21bd2 in term 1.
I20260807 18:16:40.844228 1910 leader_election.cc:302] T 941d47956d8f4eeba3646926a95ed1ba P 12b2a2e327ef42e58d42622290b21bd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12b2a2e327ef42e58d42622290b21bd2, 2ef449db5d3a489eb711bf4e769b510a; no voters:
I20260807 18:16:40.844599 2337 raft_consensus.cc:2781] T 941d47956d8f4eeba3646926a95ed1ba P 12b2a2e327ef42e58d42622290b21bd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:16:40.845124 2337 raft_consensus.cc:686] T 941d47956d8f4eeba3646926a95ed1ba P 12b2a2e327ef42e58d42622290b21bd2 [term 1 LEADER]: Becoming Leader. State: Replica: 12b2a2e327ef42e58d42622290b21bd2, State: Running, Role: LEADER
I20260807 18:16:40.845698 2337 consensus_queue.cc:237] T 941d47956d8f4eeba3646926a95ed1ba P 12b2a2e327ef42e58d42622290b21bd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef449db5d3a489eb711bf4e769b510a" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 39173 } } peers { permanent_uuid: "12b2a2e327ef42e58d42622290b21bd2" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 41407 } } peers { permanent_uuid: "e60c0179b8fa4cdbb22837c0414b5562" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 39265 } }
W20260807 18:16:40.847522 2291 tablet.cc:2369] T 941d47956d8f4eeba3646926a95ed1ba P e60c0179b8fa4cdbb22837c0414b5562: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:16:40.850837 1568 catalog_manager.cc:5495] T 941d47956d8f4eeba3646926a95ed1ba P 12b2a2e327ef42e58d42622290b21bd2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12b2a2e327ef42e58d42622290b21bd2 (127.17.110.2). New cstate: current_term: 1 leader_uuid: "12b2a2e327ef42e58d42622290b21bd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ef449db5d3a489eb711bf4e769b510a" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 39173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12b2a2e327ef42e58d42622290b21bd2" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 41407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e60c0179b8fa4cdbb22837c0414b5562" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 39265 } health_report { overall_health: UNKNOWN } } }
I20260807 18:16:40.893734 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:40.887240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55938 (local address 127.17.110.2:41407)
0807 18:16:40.887464 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:40.887470 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:40.887489 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:16:40.887590 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:40.887594 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:40.887692 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:40.887843 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:16:40.888228 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:40.889111 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:40.890771 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:40.891368 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:40.891554 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:40.892431 (+ 877us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:16:40.893050 (+ 619us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:16:40.893182 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:40.893311 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 18:16:40.893389 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 18:16:40.902801 1798 raft_consensus.cc:1261] T 941d47956d8f4eeba3646926a95ed1ba P 2ef449db5d3a489eb711bf4e769b510a [term 1 FOLLOWER]: Refusing update from remote peer 12b2a2e327ef42e58d42622290b21bd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:16:40.902841 2217 raft_consensus.cc:1261] T 941d47956d8f4eeba3646926a95ed1ba P e60c0179b8fa4cdbb22837c0414b5562 [term 1 FOLLOWER]: Refusing update from remote peer 12b2a2e327ef42e58d42622290b21bd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:16:40.903878 2342 consensus_queue.cc:1035] T 941d47956d8f4eeba3646926a95ed1ba P 12b2a2e327ef42e58d42622290b21bd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ef449db5d3a489eb711bf4e769b510a" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 39173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:16:40.904290 2337 consensus_queue.cc:1035] T 941d47956d8f4eeba3646926a95ed1ba P 12b2a2e327ef42e58d42622290b21bd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e60c0179b8fa4cdbb22837c0414b5562" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 39265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:16:40.917937 2346 mvcc.cc:205] Tried to move back new op lower bound from 7315974557287190528 to 7315974557068460032. Current Snapshot: MvccSnapshot[applied={T|T < 7315974557287190528}]
I20260807 18:16:40.919179 2350 mvcc.cc:205] Tried to move back new op lower bound from 7315974557287190528 to 7315974557068460032. Current Snapshot: MvccSnapshot[applied={T|T < 7315974557287190528}]
I20260807 18:16:40.933145 1568 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:16:40.933446 1568 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:16:40.937553 1568 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1549e73d311f40d79f3fddb57bbc9ef1: Sending DeleteTablet for 3 replicas of tablet 941d47956d8f4eeba3646926a95ed1ba
I20260807 18:16:40.938966 1775 tablet_service.cc:1514] Processing DeleteTablet for tablet 941d47956d8f4eeba3646926a95ed1ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:16:40 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:43930
I20260807 18:16:40.939090 2185 tablet_service.cc:1514] Processing DeleteTablet for tablet 941d47956d8f4eeba3646926a95ed1ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:16:40 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:42714
I20260807 18:16:40.940337 1973 tablet_service.cc:1514] Processing DeleteTablet for tablet 941d47956d8f4eeba3646926a95ed1ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:16:40 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:55930
I20260807 18:16:40.941223 2362 tablet_replica.cc:330] T 941d47956d8f4eeba3646926a95ed1ba P e60c0179b8fa4cdbb22837c0414b5562: stopping tablet replica
I20260807 18:16:40.941612 2362 raft_consensus.cc:2229] T 941d47956d8f4eeba3646926a95ed1ba P e60c0179b8fa4cdbb22837c0414b5562 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:16:40.941700 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 1684
I20260807 18:16:40.942121 2362 raft_consensus.cc:2258] T 941d47956d8f4eeba3646926a95ed1ba P e60c0179b8fa4cdbb22837c0414b5562 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:16:40.943018 2360 tablet_replica.cc:330] T 941d47956d8f4eeba3646926a95ed1ba P 2ef449db5d3a489eb711bf4e769b510a: stopping tablet replica
I20260807 18:16:40.943475 2360 raft_consensus.cc:2229] T 941d47956d8f4eeba3646926a95ed1ba P 2ef449db5d3a489eb711bf4e769b510a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:16:40.944149 2360 raft_consensus.cc:2258] T 941d47956d8f4eeba3646926a95ed1ba P 2ef449db5d3a489eb711bf4e769b510a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:16:40.945266 2363 tablet_replica.cc:330] T 941d47956d8f4eeba3646926a95ed1ba P 12b2a2e327ef42e58d42622290b21bd2: stopping tablet replica
I20260807 18:16:40.945920 2363 raft_consensus.cc:2229] T 941d47956d8f4eeba3646926a95ed1ba P 12b2a2e327ef42e58d42622290b21bd2 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:16:40.946866 2363 raft_consensus.cc:2258] T 941d47956d8f4eeba3646926a95ed1ba P 12b2a2e327ef42e58d42622290b21bd2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:16:40.948815 2360 ts_tablet_manager.cc:1905] T 941d47956d8f4eeba3646926a95ed1ba P 2ef449db5d3a489eb711bf4e769b510a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:16:40.949218 2362 ts_tablet_manager.cc:1905] T 941d47956d8f4eeba3646926a95ed1ba P e60c0179b8fa4cdbb22837c0414b5562: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:16:40.954916 2363 ts_tablet_manager.cc:1905] T 941d47956d8f4eeba3646926a95ed1ba P 12b2a2e327ef42e58d42622290b21bd2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:16:40.957485 2360 ts_tablet_manager.cc:1918] T 941d47956d8f4eeba3646926a95ed1ba P 2ef449db5d3a489eb711bf4e769b510a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:16:40.957878 2360 log.cc:1196] T 941d47956d8f4eeba3646926a95ed1ba P 2ef449db5d3a489eb711bf4e769b510a: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-0/wal/wals/941d47956d8f4eeba3646926a95ed1ba
I20260807 18:16:40.958657 2360 ts_tablet_manager.cc:1939] T 941d47956d8f4eeba3646926a95ed1ba P 2ef449db5d3a489eb711bf4e769b510a: Deleting consensus metadata
I20260807 18:16:40.960989 1549 catalog_manager.cc:4841] TS 2ef449db5d3a489eb711bf4e769b510a (127.17.110.1:39173): tablet 941d47956d8f4eeba3646926a95ed1ba (table test-table [id=aa3d09cef342442885d8fdb00da088d9]) successfully deleted
I20260807 18:16:40.961294 2362 ts_tablet_manager.cc:1918] T 941d47956d8f4eeba3646926a95ed1ba P e60c0179b8fa4cdbb22837c0414b5562: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:16:40.961515 2362 log.cc:1196] T 941d47956d8f4eeba3646926a95ed1ba P e60c0179b8fa4cdbb22837c0414b5562: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-2/wal/wals/941d47956d8f4eeba3646926a95ed1ba
I20260807 18:16:40.961977 2362 ts_tablet_manager.cc:1939] T 941d47956d8f4eeba3646926a95ed1ba P e60c0179b8fa4cdbb22837c0414b5562: Deleting consensus metadata
I20260807 18:16:40.963944 1553 catalog_manager.cc:4841] TS e60c0179b8fa4cdbb22837c0414b5562 (127.17.110.3:39265): tablet 941d47956d8f4eeba3646926a95ed1ba (table test-table [id=aa3d09cef342442885d8fdb00da088d9]) successfully deleted
I20260807 18:16:40.967121 2363 ts_tablet_manager.cc:1918] T 941d47956d8f4eeba3646926a95ed1ba P 12b2a2e327ef42e58d42622290b21bd2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:16:40.967326 2363 log.cc:1196] T 941d47956d8f4eeba3646926a95ed1ba P 12b2a2e327ef42e58d42622290b21bd2: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126429568977-17848-0/minicluster-data/ts-1/wal/wals/941d47956d8f4eeba3646926a95ed1ba
I20260807 18:16:40.968010 2363 ts_tablet_manager.cc:1939] T 941d47956d8f4eeba3646926a95ed1ba P 12b2a2e327ef42e58d42622290b21bd2: Deleting consensus metadata
I20260807 18:16:40.970208 1552 catalog_manager.cc:4841] TS 12b2a2e327ef42e58d42622290b21bd2 (127.17.110.2:41407): tablet 941d47956d8f4eeba3646926a95ed1ba (table test-table [id=aa3d09cef342442885d8fdb00da088d9]) successfully deleted
Aug 07 18:16:40 dist-test-slave-39l8 krb5kdc[868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.1@KRBTEST.COM
I20260807 18:16:40.984476 2320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:40.943155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43942 (local address 127.17.110.1:39173)
0807 18:16:40.943358 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:40.943365 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:40.959086 (+ 15721us) server_negotiation.cc:407] Connection header received
0807 18:16:40.959323 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:40.959329 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:40.959416 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:40.959531 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:40.965232 (+ 5701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:40.966432 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:40.968787 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:40.969516 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:40.969644 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:40.976034 (+ 6390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:40.976058 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:40.976080 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:40.976141 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:40.980619 (+ 4478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:40.981694 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:40.981704 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:40.981708 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:40.981932 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:40.982422 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:40.982427 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:40.982430 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:40.983118 (+ 688us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:40.983469 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:40.983886 (+ 417us) server_negotiation.cc:299] Negotiation successful
0807 18:16:40.984113 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:16:40.985314 1857 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:16:41.078964 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 1684
I20260807 18:16:41.106103 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 1874
Aug 07 18:16:41 dist-test-slave-39l8 krb5kdc[868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.2@KRBTEST.COM
I20260807 18:16:41.125931 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:41.107158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55944 (local address 127.17.110.2:41407)
0807 18:16:41.107389 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:41.107396 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:41.107417 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:16:41.107489 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:41.107494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:41.107577 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:41.107725 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:41.109124 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:41.110488 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:41.112074 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:41.112805 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:41.112960 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:41.117785 (+ 4825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:41.117804 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:41.117826 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:41.117901 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:41.122404 (+ 4503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:41.123398 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:41.123409 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:41.123413 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:41.123631 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:41.124048 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:41.124053 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:41.124057 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:41.124574 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:41.124944 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:41.125227 (+ 283us) server_negotiation.cc:299] Negotiation successful
0807 18:16:41.125483 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 18:16:41.126890 2070 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:16:41.247725 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 1874
I20260807 18:16:41.274570 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 2090
Aug 07 18:16:41 dist-test-slave-39l8 krb5kdc[868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.3@KRBTEST.COM
I20260807 18:16:41.291211 2327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:41.275574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42716 (local address 127.17.110.3:39265)
0807 18:16:41.275765 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:41.275773 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:41.275856 (+ 83us) server_negotiation.cc:407] Connection header received
0807 18:16:41.276075 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:41.276080 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:41.276158 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:41.276274 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:41.277695 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:41.278427 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:41.279809 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:41.280212 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:41.280360 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:41.285134 (+ 4774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:41.285153 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:41.285171 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:41.285216 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:41.288133 (+ 2917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:41.288947 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:41.288955 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:41.288957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:41.289100 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:41.289489 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:41.289494 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:41.289497 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:41.289913 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:41.290216 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:41.290557 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 18:16:41.290840 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 18:16:41.291949 2281 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:16:41.384367 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 2090
I20260807 18:16:41.410032 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 1526
I20260807 18:16:41.411346 1613 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:16:41.532899 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 1526
I20260807 18:16:41.571760 17848 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 18:16:41.918394 17848 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 18:16:41.918687 17848 mini_ranger.cc:66] Stopping Ranger...
I20260807 18:16:42.276013 17848 mini_ranger.cc:68] Stopped Ranger
2026-08-07 18:16:42.276 UTC [915] LOG: received smart shutdown request
2026-08-07 18:16:42.280 UTC [915] LOG: background worker "logical replication launcher" (PID 928) exited with exit code 1
2026-08-07 18:16:42.280 UTC [922] LOG: shutting down
2026-08-07 18:16:42.454 UTC [915] LOG: database system is shut down
2026-08-07T18:16:42Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (57459 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:16:42 dist-test-slave-39l8 krb5kdc[2385](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:16:42 dist-test-slave-39l8 krb5kdc[2385](info): set up 2 sockets
Aug 07 18:16:42 dist-test-slave-39l8 krb5kdc[2385](info): commencing operation
krb5kdc: starting...
W20260807 18:16:43.951635 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.297s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:16:44 dist-test-slave-39l8 krb5kdc[2385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:16:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:16:44Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.17.110.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.17.110.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.17.110.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/rangeradmin_127.17.110.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.17.110.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/rangeradmin_127.17.110.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.17.110.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.17.110.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.17.110.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/rangerlookup_127.17.110.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.17.110.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/rangerlookup_127.17.110.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.110.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.17.110.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.110.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/HTTP_127.17.110.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.110.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/HTTP_127.17.110.20@KRBTEST.COM.keytab.
I20260807 18:16:44.390442 17848 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskDgKS9Z/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/postgres -l logfile start
2026-08-07 18:16:47.026 UTC [2427] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 18:16:47.026 UTC [2427] LOG: listening on IPv4 address "127.17.110.20", port 33815
2026-08-07 18:16:47.030 UTC [2427] LOG: listening on Unix socket "/tmp/.s.PGSQL.33815"
2026-08-07 18:16:47.054 UTC [2431] LOG: database system was shut down at 2026-08-07 18:16:45 UTC
2026-08-07 18:16:47.058 UTC [2427] LOG: database system is ready to accept connections
I20260807 18:16:48.251003 17848 mini_postgres.cc:96] Postgres bound to 33815
2026-08-07 18:16:48.278 UTC [2446] FATAL: database "slave" does not exist
127.17.110.20:33815 - accepting connections
I20260807 18:16:48.279350 17848 mini_ranger.cc:161] Starting Ranger...
I20260807 18:16:48.316550 17848 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 18:16:48.849350 17848 mini_ranger.cc:87] Created ranger Postgres database
I20260807 18:16:48.849511 17848 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ranger-admin
2026-08-07 18:16:49,157 [I] DB FLAVOR :POSTGRES
2026-08-07 18:16:49,157 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 18:16:49,158 [I] Checking connection..
2026-08-07 18:16:49,158 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:33815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 18:16:49,446 [I] Checking connection passed.
2026-08-07 18:16:49,447 [I] --------- Verifying version history table ---------
2026-08-07 18:16:49,447 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:33815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:16:49,749 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 18:16:49,749 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:33815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:16:50,039 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 18:16:50,040 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 18:16:50,040 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:33815/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 18:16:50.320 UTC [2547] WARNING: there is no transaction in progress
2026-08-07 18:16:50,329 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 18:16:50,330 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:33815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:16:50,595 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 18:16:50,595 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 18:16:50,595 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:33815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 18:16:50,857 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:33815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 18:16:51,107 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:33815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-39l8.c.gcp-upstream.internal','N') ;"
2026-08-07 18:16:51,410 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:33815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 18:16:51,700 [I] Table x_portal_user does not exist in database ranger
2026-08-07 18:16:51,700 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:33815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 18:16:51,978 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 18:16:51,978 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:33815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 18:16:52,273 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:33815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 18:16:52,557 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 18:16:52,558 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:33815/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 18:16:53.781 UTC [2719] WARNING: there is no transaction in progress
2026-08-07 18:16:53.801 UTC [2719] WARNING: there is no transaction in progress
2026-08-07 18:16:53.823 UTC [2719] WARNING: there is no transaction in progress
2026-08-07 18:16:53.843 UTC [2719] WARNING: there is no transaction in progress
2026-08-07 18:16:53.863 UTC [2719] WARNING: there is no transaction in progress
2026-08-07 18:16:53.886 UTC [2719] WARNING: there is no transaction in progress
2026-08-07 18:16:53.953 UTC [2719] WARNING: there is no transaction in progress
2026-08-07 18:16:53.961 UTC [2719] WARNING: there is no transaction in progress
2026-08-07 18:16:53.979 UTC [2719] WARNING: there is no transaction in progress
2026-08-07 18:16:53.991 UTC [2719] WARNING: there is no transaction in progress
2026-08-07 18:16:54.004 UTC [2719] WARNING: there is no transaction in progress
2026-08-07 18:16:54.022 UTC [2719] WARNING: there is no transaction in progress
2026-08-07 18:16:54.033 UTC [2719] WARNING: there is no transaction in progress
2026-08-07 18:16:54.041 UTC [2719] WARNING: there is no transaction in progress
2026-08-07 18:16:54.050 UTC [2719] WARNING: there is no transaction in progress
2026-08-07 18:16:54.058 UTC [2719] WARNING: there is no transaction in progress
2026-08-07 18:16:54,837 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 18:16:54,838 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:33815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 18:16:55,130 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 18:16:55,130 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:33815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 18:16:55,435 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 18:16:55,436 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:33815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 18:16:55,714 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 18:16:55,714 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:33815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 18:16:56,003 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:33815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 18:16:56,278 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:33815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-39l8.c.gcp-upstream.internal';"
2026-08-07 18:16:56,533 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 18:16:56,533 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:33815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 18:16:56,843 [I] DB_PATCHES have already been applied
I20260807 18:16:56.850097 17848 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ranger-admin:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskDgKS9Z/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 18:16:56.850262 17848 mini_ranger.cc:193] Using host: 127.17.110.20
I20260807 18:16:56.872774 17848 mini_ranger.cc:227] Ranger admin URL: http://127.17.110.20:53287
Aug 07, 2026 6:16:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:16:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:16:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 6:16:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 6:16:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 6:16:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 6:16:57 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 18:16:58.371424 17848 mini_ranger.cc:160] Time spent starting Ranger: real 10.092s user 0.010s sys 0.061s
Aug 07, 2026 6:17:22 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 6:17:22 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 07, 2026 6:17:22 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 6:17:22 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 6:17:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 6:17:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 6:17:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 6:17:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 6:17:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 6:17:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 6:17:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 6:17:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 6:17:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 6:17:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 6:17:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 6:17:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 6:17:22 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 6:17:23 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 18:17:24.361610 17848 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.17.110.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.17.110.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.17.110.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/rangerkms_127.17.110.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.17.110.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/rangerkms_127.17.110.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.110.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.17.110.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.110.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/HTTP_127.17.110.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.110.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/HTTP_127.17.110.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 18:17:24 dist-test-slave-39l8 krb5kdc[2385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126644, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 18:17:24.573014 17848 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 18:17:24.603263 17848 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 18:17:25.530800 17848 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 18:17:25.530927 17848 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ranger-kms
I20260807 18:17:25.530982 17848 mini_ranger_kms.cc:227] Using postgres at 127.17.110.20:33815
2026-08-07 18:17:25,917 [I] DB FLAVOR :POSTGRES
2026-08-07 18:17:25,917 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 18:17:25,918 [I] Checking connection
2026-08-07 18:17:25,918 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:33815/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 18:17:26,214 [I] connection success
2026-08-07 18:17:26,215 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 18:17:26,215 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 18:17:26,215 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:33815/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 18:17:26,481 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 18:17:26,481 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 18:17:26,481 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 18:17:26,482 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.110.20:33815/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 18:17:26,760 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 18:17:27.332506 17848 mini_ranger_kms.cc:310] Created kms service
I20260807 18:17:27.565866 17848 mini_ranger_kms.cc:326] Created kudu user
I20260807 18:17:27.623340 17848 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 18:17:27 dist-test-slave-39l8 krb5kdc[2385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126644, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.17.110.20@KRBTEST.COM
I20260807 18:17:28.072646 17848 mini_ranger_kms.cc:384] Added ranger policy
I20260807 18:17:28.072988 17848 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ranger-kms:/tmp/dist-test-taskDgKS9Z/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskDgKS9Z/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 18:17:28.073032 17848 mini_ranger_kms.cc:240] Using host: 127.17.110.20
I20260807 18:17:28.099800 17848 mini_ranger_kms.cc:276] Ranger KMS PID: 2965
I20260807 18:17:28.099963 17848 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.17.110.20:43015
Aug 07, 2026 6:17:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:17:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:17:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 6:17:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 6:17:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 6:17:29 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 18:17:30 dist-test-slave-39l8 krb5kdc[2385](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786126650, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.110.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:30.989310 17848 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.416s user 0.061s sys 0.358s
I20260807 18:17:30.989607 17848 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 18:17:30.989948 17848 mini_ranger_kms.cc:401] 127.17.110.20:43015/kms/v1/keys
Aug 07 18:17:30 dist-test-slave-39l8 krb5kdc[2385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126650, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.110.20@KRBTEST.COM for HTTP/127.17.110.20@KRBTEST.COM
Aug 07 18:17:31 dist-test-slave-39l8 krb5kdc[2385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126650, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.110.20@KRBTEST.COM for HTTP/127.17.110.20@KRBTEST.COM
Aug 07 18:17:33 dist-test-slave-39l8 krb5kdc[2385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.17.110.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
I20260807 18:17:33.966507 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:40945
--webserver_interface=127.17.110.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:37857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:40945
--ranger_config_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.110.20:43015/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:17:34.342159 3035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:34.342555 3035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:34.342795 3035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:34.350508 3035 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:17:34.350693 3035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:34.350824 3035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:34.350934 3035 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:17:34.351035 3035 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:17:34.362183 3035 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:37857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/master-0/wal
--ranger_kms_url=127.17.110.20:43015/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:40945
--ranger_config_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:40945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:17:34.363770 3035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:34.365273 3035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:34.378687 3045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:34.378913 3041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:34.378882 3042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:34.379808 3035 server_base.cc:1034] running on GCE node
I20260807 18:17:34.380831 3035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:34.382798 3035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:34.384061 3035 hybrid_clock.cc:648] HybridClock initialized: now 1786126654384034 us; error 71 us; skew 500 ppm
I20260807 18:17:34.384588 3035 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:34 dist-test-slave-39l8 krb5kdc[2385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126654, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:34.388546 3035 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:17:34.390393 3035 webserver.cc:460] Webserver started at http://127.17.110.62:45701/ using document root <none> and password file <none>
I20260807 18:17:34.391234 3035 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:34.391382 3035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:34.391767 3035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:17:34 dist-test-slave-39l8 krb5kdc[2385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126654, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for HTTP/127.17.110.20@KRBTEST.COM
I20260807 18:17:34.455593 3035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "5598f28d1d0343d19ab15850f27a752c"
format_stamp: "Formatted at 2026-08-07 18:17:34 on dist-test-slave-39l8"
server_key: "f9ba3fdb4dfe5629f357bb5808e4c815"
server_key_iv: "9d8824a3d80bc9e48d93e9a61bc020ab"
server_key_version: "kuduclusterkey@0"
I20260807 18:17:34.456570 3035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "5598f28d1d0343d19ab15850f27a752c"
format_stamp: "Formatted at 2026-08-07 18:17:34 on dist-test-slave-39l8"
server_key: "f9ba3fdb4dfe5629f357bb5808e4c815"
server_key_iv: "9d8824a3d80bc9e48d93e9a61bc020ab"
server_key_version: "kuduclusterkey@0"
I20260807 18:17:34.462790 3035 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.004s
I20260807 18:17:34.484437 3055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:34.486064 3035 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:17:34.486286 3035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "5598f28d1d0343d19ab15850f27a752c"
format_stamp: "Formatted at 2026-08-07 18:17:34 on dist-test-slave-39l8"
server_key: "f9ba3fdb4dfe5629f357bb5808e4c815"
server_key_iv: "9d8824a3d80bc9e48d93e9a61bc020ab"
server_key_version: "kuduclusterkey@0"
I20260807 18:17:34.486536 3035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:34.503926 3035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:34.508656 3035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:34.509037 3035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:34.530431 3035 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:40945
I20260807 18:17:34.530470 3132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:40945 every 8 connection(s)
I20260807 18:17:34.532047 3035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:17:34.534524 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 3035
I20260807 18:17:34.534827 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:17:34.535526 17848 external_mini_cluster.cc:1428] Setting key d39015f167d47c03d97d917222cee23f
I20260807 18:17:34.536845 3133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 18:17:34 dist-test-slave-39l8 krb5kdc[2385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:17:34.550853 3133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c: Bootstrap starting.
I20260807 18:17:34.554935 3133 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:34.556233 3133 log.cc:824] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:34.559989 3133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c: No bootstrap required, opened a new log
I20260807 18:17:34.560307 3137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:34.537742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47494 (local address 127.17.110.62:40945)
0807 18:17:34.538870 (+ 1128us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:34.538892 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:34.538954 (+ 62us) server_negotiation.cc:407] Connection header received
0807 18:17:34.540754 (+ 1800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:34.540780 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:34.541271 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:34.541740 (+ 469us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:34.543428 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:34.544841 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:34.546257 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:34.547101 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:34.547339 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:34.551616 (+ 4277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:34.551626 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:34.551643 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:34.551693 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:34.555279 (+ 3586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:34.556131 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:34.556138 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:34.556146 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:34.556320 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:34.556759 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:34.556763 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:34.556765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:34.557647 (+ 882us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:34.557960 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:34.558536 (+ 576us) server_negotiation.cc:299] Negotiation successful
0807 18:17:34.559221 (+ 685us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":714,"thread_start_us":467,"threads_started":1}
I20260807 18:17:34.568702 3133 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5598f28d1d0343d19ab15850f27a752c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40945 } }
I20260807 18:17:34.569175 3133 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:34.569316 3133 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5598f28d1d0343d19ab15850f27a752c, State: Initialized, Role: FOLLOWER
I20260807 18:17:34.569957 3133 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5598f28d1d0343d19ab15850f27a752c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40945 } }
I20260807 18:17:34.570231 3133 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:17:34.570398 3133 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:17:34.570598 3133 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:34.573357 3133 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5598f28d1d0343d19ab15850f27a752c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40945 } }
I20260807 18:17:34.573980 3133 leader_election.cc:302] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5598f28d1d0343d19ab15850f27a752c; no voters:
I20260807 18:17:34.574651 3133 leader_election.cc:288] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:17:34.574867 3140 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:17:34.576536 3140 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [term 1 LEADER]: Becoming Leader. State: Replica: 5598f28d1d0343d19ab15850f27a752c, State: Running, Role: LEADER
I20260807 18:17:34.577361 3133 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:17:34.577257 3140 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5598f28d1d0343d19ab15850f27a752c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40945 } }
I20260807 18:17:34.581194 3141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5598f28d1d0343d19ab15850f27a752c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5598f28d1d0343d19ab15850f27a752c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40945 } } }
I20260807 18:17:34.581136 3142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5598f28d1d0343d19ab15850f27a752c. Latest consensus state: current_term: 1 leader_uuid: "5598f28d1d0343d19ab15850f27a752c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5598f28d1d0343d19ab15850f27a752c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40945 } } }
I20260807 18:17:34.581626 3142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [sys.catalog]: This master's current role is: LEADER
I20260807 18:17:34.581624 3141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [sys.catalog]: This master's current role is: LEADER
I20260807 18:17:34.583700 3152 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:17:34.589658 3152 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:17:34.590545 3133 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 18:17:34.611300 3152 catalog_manager.cc:1347] Generated new cluster ID: 921f81dc1f8840c9ba853f2022ec8f0f
I20260807 18:17:34.611477 3152 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:17:34.633265 3152 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:17:34.635406 3152 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:17:34.660279 3152 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c: Generated new TSK 0
I20260807 18:17:34.661550 3152 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.110.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
I20260807 18:17:35.502331 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.110.20:43015/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.1:0
--local_ip_for_outbound_sockets=127.17.110.1
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:40945
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:37857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:17:35 dist-test-slave-39l8 krb5kdc[2385](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786126655, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 18:17:36.008150 3196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:36.008715 3196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:36.008927 3196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:36.020589 3196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:36.020754 3196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:36.020959 3196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.1
I20260807 18:17:36.031929 3196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:37857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.17.110.20:43015/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:40945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.1
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:17:36.033452 3196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:36.035006 3196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:36.051231 3215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:36.054277 3217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:36.055078 3213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:36.057099 3196 server_base.cc:1034] running on GCE node
I20260807 18:17:36.058020 3196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:36.059336 3196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:36.060672 3196 hybrid_clock.cc:648] HybridClock initialized: now 1786126656060556 us; error 103 us; skew 500 ppm
I20260807 18:17:36.061141 3196 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:36 dist-test-slave-39l8 krb5kdc[2385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126656, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:36.065860 3196 init.cc:377] Logged in from keytab as kudu/127.17.110.1@KRBTEST.COM (short username kudu)
I20260807 18:17:36.069265 3196 webserver.cc:460] Webserver started at http://127.17.110.1:40275/ using document root <none> and password file <none>
I20260807 18:17:36.070137 3196 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:36.070317 3196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:36.070749 3196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:17:36 dist-test-slave-39l8 krb5kdc[2385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126656, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for HTTP/127.17.110.20@KRBTEST.COM
I20260807 18:17:36.108609 3196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-0/data/instance:
uuid: "5fea522d5d5a4017b703d7e35d9a4f7f"
format_stamp: "Formatted at 2026-08-07 18:17:36 on dist-test-slave-39l8"
server_key: "f28283c4a86f00397555772f798b4288"
server_key_iv: "640e6413049891b01a2917a3ac6519eb"
server_key_version: "kuduclusterkey@0"
I20260807 18:17:36.110909 3196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance:
uuid: "5fea522d5d5a4017b703d7e35d9a4f7f"
format_stamp: "Formatted at 2026-08-07 18:17:36 on dist-test-slave-39l8"
server_key: "f28283c4a86f00397555772f798b4288"
server_key_iv: "640e6413049891b01a2917a3ac6519eb"
server_key_version: "kuduclusterkey@0"
I20260807 18:17:36.136092 3196 fs_manager.cc:691] Time spent creating directory manager: real 0.024s user 0.014s sys 0.005s
I20260807 18:17:36.171653 3229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:36.174111 3196 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.004s sys 0.008s
I20260807 18:17:36.174504 3196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-0/wal
uuid: "5fea522d5d5a4017b703d7e35d9a4f7f"
format_stamp: "Formatted at 2026-08-07 18:17:36 on dist-test-slave-39l8"
server_key: "f28283c4a86f00397555772f798b4288"
server_key_iv: "640e6413049891b01a2917a3ac6519eb"
server_key_version: "kuduclusterkey@0"
I20260807 18:17:36.174995 3196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:36.237895 3196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:36.244807 3196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:36.245266 3196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:36.247051 3196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:17:36.249742 3196 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:17:36.249884 3196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:36.250108 3196 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:17:36.250203 3196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:17:36 dist-test-slave-39l8 krb5kdc[2385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126655, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for HTTP/127.17.110.20@KRBTEST.COM
I20260807 18:17:36.489554 3196 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.1:36337
I20260807 18:17:36.489617 3411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.1:36337 every 8 connection(s)
I20260807 18:17:36.491143 3196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
I20260807 18:17:36.497925 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 3196
I20260807 18:17:36.498226 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance
I20260807 18:17:36.499377 17848 external_mini_cluster.cc:1428] Setting key d8a8a9ee82452a135f7f5d0553a168a2
Aug 07 18:17:36 dist-test-slave-39l8 krb5kdc[2385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126656, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.110.2@KRBTEST.COM; defaulting to no policy
I20260807 18:17:36.540448 3414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:36.494807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:58629 (local address 127.17.110.62:40945)
0807 18:17:36.507287 (+ 12480us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:36.507298 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:36.510847 (+ 3549us) server_negotiation.cc:407] Connection header received
0807 18:17:36.511274 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:36.511283 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:36.511410 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:36.511548 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:36.514034 (+ 2486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:36.515355 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:36.517230 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:36.517684 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:36.517855 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:36.523544 (+ 5689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:36.523569 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:36.523578 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:36.523639 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:36.528913 (+ 5274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:36.530387 (+ 1474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:36.530396 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:36.530403 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:36.530628 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:36.531421 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:36.531427 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:36.531432 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:36.531847 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:36.531995 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:36.539658 (+ 7663us) server_negotiation.cc:299] Negotiation successful
0807 18:17:36.539991 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12277,"thread_start_us":12111,"threads_started":1}
I20260807 18:17:36.541699 3416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:36.495248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40945 (local address 127.17.110.1:58629)
0807 18:17:36.507883 (+ 12635us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:36.507957 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:36.510845 (+ 2888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:36.511928 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:36.511943 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:36.512716 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:36.513737 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:36.513770 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:36.515591 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:36.515596 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:36.517013 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:36.517030 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:36.518006 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:36.518017 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:36.518286 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:36.519252 (+ 966us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:17:36.519290 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:36.523141 (+ 3851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:36.529243 (+ 6102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:36.529258 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:36.529284 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:36.530148 (+ 864us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:36.530836 (+ 688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:36.530842 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:36.530846 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:36.531259 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:36.532127 (+ 868us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:36.532137 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:36.532674 (+ 537us) client_negotiation.cc:769] Sending connection context
0807 18:17:36.533027 (+ 353us) client_negotiation.cc:240] Negotiation successful
0807 18:17:36.533655 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12236,"thread_start_us":11995,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.2@KRBTEST.COM" created.
I20260807 18:17:36.546584 3412 heartbeater.cc:347] Connected to a master server at 127.17.110.62:40945
I20260807 18:17:36.547063 3412 heartbeater.cc:464] Registering TS with master...
I20260807 18:17:36.547879 3412 heartbeater.cc:511] Master 127.17.110.62:40945 requested a full tablet report, sending...
I20260807 18:17:36.550860 3080 ts_manager.cc:194] Registered new tserver with Master: 5fea522d5d5a4017b703d7e35d9a4f7f (127.17.110.1:36337)
I20260807 18:17:36.555308 3080 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.1@KRBTEST.COM'} at 127.17.110.1:58629
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
I20260807 18:17:36.590879 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.110.20:43015/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.2:0
--local_ip_for_outbound_sockets=127.17.110.2
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:40945
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:37857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:17:36 dist-test-slave-39l8 krb5kdc[2385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126655, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for HTTP/127.17.110.20@KRBTEST.COM
W20260807 18:17:37.182654 3423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:37.183132 3423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:37.183337 3423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:37.191656 3423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:37.191854 3423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:37.192063 3423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.2
I20260807 18:17:37.203114 3423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:37857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.17.110.20:43015/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:40945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.2
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:17:37.205060 3423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:37.207206 3423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:37.231488 3448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:37.239284 3449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:37.242333 3423 server_base.cc:1034] running on GCE node
W20260807 18:17:37.243005 3454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:37.244120 3423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:37.245628 3423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:37.246963 3423 hybrid_clock.cc:648] HybridClock initialized: now 1786126657246890 us; error 67 us; skew 500 ppm
I20260807 18:17:37.247416 3423 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:37 dist-test-slave-39l8 krb5kdc[2385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126657, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:37.253202 3423 init.cc:377] Logged in from keytab as kudu/127.17.110.2@KRBTEST.COM (short username kudu)
I20260807 18:17:37.256656 3423 webserver.cc:460] Webserver started at http://127.17.110.2:42759/ using document root <none> and password file <none>
I20260807 18:17:37.257916 3423 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:37.258131 3423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:37.258608 3423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:17:37 dist-test-slave-39l8 krb5kdc[2385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126657, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for HTTP/127.17.110.20@KRBTEST.COM
I20260807 18:17:37.284245 3423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-1/data/instance:
uuid: "84a010898a41411189392239eefdc0d2"
format_stamp: "Formatted at 2026-08-07 18:17:37 on dist-test-slave-39l8"
server_key: "f8f3e230cda5ccc8316dd3cd3be90acd"
server_key_iv: "338d9e179d49f234a1086b48be5785b0"
server_key_version: "kuduclusterkey@0"
I20260807 18:17:37.286062 3423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance:
uuid: "84a010898a41411189392239eefdc0d2"
format_stamp: "Formatted at 2026-08-07 18:17:37 on dist-test-slave-39l8"
server_key: "f8f3e230cda5ccc8316dd3cd3be90acd"
server_key_iv: "338d9e179d49f234a1086b48be5785b0"
server_key_version: "kuduclusterkey@0"
I20260807 18:17:37.309015 3423 fs_manager.cc:691] Time spent creating directory manager: real 0.022s user 0.003s sys 0.014s
I20260807 18:17:37.340649 3463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:37.344493 3423 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.004s sys 0.000s
I20260807 18:17:37.345062 3423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-1/wal
uuid: "84a010898a41411189392239eefdc0d2"
format_stamp: "Formatted at 2026-08-07 18:17:37 on dist-test-slave-39l8"
server_key: "f8f3e230cda5ccc8316dd3cd3be90acd"
server_key_iv: "338d9e179d49f234a1086b48be5785b0"
server_key_version: "kuduclusterkey@0"
I20260807 18:17:37.345479 3423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:37.399849 3423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:37.406407 3423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:37.406921 3423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:37.414594 3423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:17:37.423763 3423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:17:37.423992 3423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 18:17:37.424189 3423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:17:37.424304 3423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:37.485225 3423 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.2:33205
I20260807 18:17:37.485294 3639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.2:33205 every 8 connection(s)
I20260807 18:17:37.487274 3423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
I20260807 18:17:37.487768 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 3423
I20260807 18:17:37.487965 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance
I20260807 18:17:37.488591 17848 external_mini_cluster.cc:1428] Setting key d2d9c81ae78fe6e21b47f9e711c320e7
Aug 07 18:17:37 dist-test-slave-39l8 krb5kdc[2385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126657, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.110.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.3@KRBTEST.COM" created.
I20260807 18:17:37.552376 3648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:37.516738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:53021 (local address 127.17.110.62:40945)
0807 18:17:37.525475 (+ 8737us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:37.525482 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:37.529960 (+ 4478us) server_negotiation.cc:407] Connection header received
0807 18:17:37.530071 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:37.530077 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:37.530275 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:37.530392 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:37.533260 (+ 2868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:37.534617 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:37.537456 (+ 2839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:37.538189 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:37.538305 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:37.543429 (+ 5124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:37.543438 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:37.543443 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:37.543500 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:37.547305 (+ 3805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:37.548663 (+ 1358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:37.548670 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:37.548674 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:37.548870 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:37.549664 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:37.549671 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:37.549675 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:37.550138 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:37.550325 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:37.551554 (+ 1229us) server_negotiation.cc:299] Negotiation successful
0807 18:17:37.551919 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8588,"thread_start_us":8440,"threads_started":1}
I20260807 18:17:37.559684 3412 heartbeater.cc:503] Master 127.17.110.62:40945 was elected leader, sending a full tablet report...
I20260807 18:17:37.561126 3646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:37.515919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40945 (local address 127.17.110.2:53021)
0807 18:17:37.526224 (+ 10305us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:37.526307 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:37.529372 (+ 3065us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:37.531025 (+ 1653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:37.531044 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:37.531766 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:37.532696 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:37.532725 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:37.534942 (+ 2217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:37.534948 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:37.537214 (+ 2266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:37.537237 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:37.538785 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:37.538794 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:37.539154 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:37.540093 (+ 939us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:17:37.540127 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:37.543092 (+ 2965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:37.547571 (+ 4479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:37.547583 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:37.547606 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:37.548363 (+ 757us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:37.549047 (+ 684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:37.549053 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:37.549056 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:37.549428 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:37.550443 (+ 1015us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:37.550453 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:37.551012 (+ 559us) client_negotiation.cc:769] Sending connection context
0807 18:17:37.552732 (+ 1720us) client_negotiation.cc:240] Negotiation successful
0807 18:17:37.559521 (+ 6789us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9861,"thread_start_us":10658,"threads_started":1}
I20260807 18:17:37.563417 3640 heartbeater.cc:347] Connected to a master server at 127.17.110.62:40945
I20260807 18:17:37.563805 3640 heartbeater.cc:464] Registering TS with master...
I20260807 18:17:37.564733 3640 heartbeater.cc:511] Master 127.17.110.62:40945 requested a full tablet report, sending...
I20260807 18:17:37.566764 3080 ts_manager.cc:194] Registered new tserver with Master: 84a010898a41411189392239eefdc0d2 (127.17.110.2:33205)
I20260807 18:17:37.567950 3080 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.2@KRBTEST.COM'} at 127.17.110.2:53021
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
W20260807 18:17:37.600724 17848 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.17.110.3: real 0.103s user 0.000s sys 0.024s
I20260807 18:17:37.601781 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.110.20:43015/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.3:0
--local_ip_for_outbound_sockets=127.17.110.3
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:40945
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:37857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:17:38.011798 3650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:38.012295 3650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:38.012531 3650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:38.024507 3650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:38.024685 3650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:38.024880 3650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.3
I20260807 18:17:38.035876 3650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:37857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.17.110.20:43015/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:40945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.3
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:17:38.037743 3650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:38.039285 3650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:38.054394 3657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:38.054319 3659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:38.055076 3661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:38.060563 3650 server_base.cc:1034] running on GCE node
I20260807 18:17:38.061379 3650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:38.062644 3650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:38.063957 3650 hybrid_clock.cc:648] HybridClock initialized: now 1786126658063881 us; error 68 us; skew 500 ppm
I20260807 18:17:38.064363 3650 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:38 dist-test-slave-39l8 krb5kdc[2385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126658, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:38.068076 3650 init.cc:377] Logged in from keytab as kudu/127.17.110.3@KRBTEST.COM (short username kudu)
I20260807 18:17:38.070065 3650 webserver.cc:460] Webserver started at http://127.17.110.3:34549/ using document root <none> and password file <none>
I20260807 18:17:38.070945 3650 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:38.071089 3650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:38.071476 3650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:17:38 dist-test-slave-39l8 krb5kdc[2385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126658, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for HTTP/127.17.110.20@KRBTEST.COM
I20260807 18:17:38.092067 3650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-2/data/instance:
uuid: "71b7ee428f154c388f33cd1ade59f13a"
format_stamp: "Formatted at 2026-08-07 18:17:38 on dist-test-slave-39l8"
server_key: "3251a48f0567e0d6596ca2b4b23ad556"
server_key_iv: "35ffb5147823cbbebfa277dc81412b2d"
server_key_version: "kuduclusterkey@0"
I20260807 18:17:38.093418 3650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance:
uuid: "71b7ee428f154c388f33cd1ade59f13a"
format_stamp: "Formatted at 2026-08-07 18:17:38 on dist-test-slave-39l8"
server_key: "3251a48f0567e0d6596ca2b4b23ad556"
server_key_iv: "35ffb5147823cbbebfa277dc81412b2d"
server_key_version: "kuduclusterkey@0"
I20260807 18:17:38.101907 3650 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s
I20260807 18:17:38.122697 3670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:38.124783 3650 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 18:17:38.124991 3650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-2/wal
uuid: "71b7ee428f154c388f33cd1ade59f13a"
format_stamp: "Formatted at 2026-08-07 18:17:38 on dist-test-slave-39l8"
server_key: "3251a48f0567e0d6596ca2b4b23ad556"
server_key_iv: "35ffb5147823cbbebfa277dc81412b2d"
server_key_version: "kuduclusterkey@0"
I20260807 18:17:38.125264 3650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:38.151497 3650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:38.157138 3650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:38.157522 3650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:38.158890 3650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:17:38.160804 3650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:17:38.160913 3650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:38.161098 3650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:17:38.161170 3650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:38.216562 3650 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.3:41459
I20260807 18:17:38.216609 3841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.3:41459 every 8 connection(s)
I20260807 18:17:38.218446 3650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
I20260807 18:17:38.223246 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 3650
I20260807 18:17:38.223439 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance
I20260807 18:17:38.224056 17848 external_mini_cluster.cc:1428] Setting key 187b8ea52f4dcafc7346889e9810ff7c
Aug 07 18:17:38 dist-test-slave-39l8 krb5kdc[2385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126658, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:17:38.261725 3845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:38.221605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:56955 (local address 127.17.110.62:40945)
0807 18:17:38.231247 (+ 9642us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:38.231258 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:38.237838 (+ 6580us) server_negotiation.cc:407] Connection header received
0807 18:17:38.241454 (+ 3616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:38.241460 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:38.241575 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:38.241705 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:38.243887 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.245034 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.247666 (+ 2632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.248332 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.248477 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:38.253715 (+ 5238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:38.253722 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:38.253727 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:38.253781 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:38.257505 (+ 3724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:38.258678 (+ 1173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:38.258684 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:38.258688 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:38.258944 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:38.259553 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:38.259559 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:38.259562 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:38.259980 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:38.260155 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:38.261091 (+ 936us) server_negotiation.cc:299] Negotiation successful
0807 18:17:38.261344 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9439,"thread_start_us":9278,"threads_started":1}
I20260807 18:17:38.262933 3846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:38.222349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40945 (local address 127.17.110.3:56955)
0807 18:17:38.237555 (+ 15206us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:38.237641 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:38.241067 (+ 3426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:38.242051 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:38.242062 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:38.242759 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:38.243653 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:38.243681 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.245237 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.245243 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:38.247437 (+ 2194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:38.247456 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.248640 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.248649 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:38.248996 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:38.250055 (+ 1059us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:17:38.250089 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:38.253415 (+ 3326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:38.257760 (+ 4345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:38.257772 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:38.257794 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:38.258489 (+ 695us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:38.259084 (+ 595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:38.259089 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:38.259093 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:38.259403 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:38.260137 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:38.260146 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:38.260603 (+ 457us) client_negotiation.cc:769] Sending connection context
0807 18:17:38.260933 (+ 330us) client_negotiation.cc:240] Negotiation successful
0807 18:17:38.261495 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14813,"thread_start_us":15415,"threads_started":1}
I20260807 18:17:38.264998 3843 heartbeater.cc:347] Connected to a master server at 127.17.110.62:40945
I20260807 18:17:38.265357 3843 heartbeater.cc:464] Registering TS with master...
I20260807 18:17:38.266283 3843 heartbeater.cc:511] Master 127.17.110.62:40945 requested a full tablet report, sending...
I20260807 18:17:38.268187 3080 ts_manager.cc:194] Registered new tserver with Master: 71b7ee428f154c388f33cd1ade59f13a (127.17.110.3:41459)
I20260807 18:17:38.269310 3080 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.3@KRBTEST.COM'} at 127.17.110.3:56955
I20260807 18:17:38.279309 17848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:17:38.281911 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 3196
Aug 07 18:17:38 dist-test-slave-39l8 krb5kdc[2385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.1@KRBTEST.COM
I20260807 18:17:38.317766 3849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:38.283180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60374 (local address 127.17.110.1:36337)
0807 18:17:38.295600 (+ 12420us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:38.295610 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:38.295656 (+ 46us) server_negotiation.cc:407] Connection header received
0807 18:17:38.295736 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:38.295745 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:38.296055 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:38.298209 (+ 2154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:38.298517 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.300132 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.302326 (+ 2194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.303288 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.303427 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:38.309601 (+ 6174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:38.309634 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:38.309656 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:38.309711 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:38.313794 (+ 4083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:38.314902 (+ 1108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:38.314912 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:38.314916 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:38.315144 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:38.315679 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:38.315683 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:38.315686 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:38.316223 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:38.316575 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:38.317003 (+ 428us) server_negotiation.cc:299] Negotiation successful
0807 18:17:38.317334 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12217,"thread_start_us":12048,"threads_started":1}
I20260807 18:17:38.319653 3401 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:38.463479 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 3196
I20260807 18:17:38.497267 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 3423
Aug 07 18:17:38 dist-test-slave-39l8 krb5kdc[2385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.2@KRBTEST.COM
I20260807 18:17:38.522085 3851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:38.499136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58268 (local address 127.17.110.2:33205)
0807 18:17:38.499966 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:38.499975 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:38.500006 (+ 31us) server_negotiation.cc:407] Connection header received
0807 18:17:38.500083 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:38.500091 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:38.500372 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:38.502292 (+ 1920us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:38.502578 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.504379 (+ 1801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.506467 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.507461 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.508343 (+ 882us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:38.513813 (+ 5470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:38.513835 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:38.513859 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:38.513917 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:38.518316 (+ 4399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:38.519473 (+ 1157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:38.519484 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:38.519488 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:38.519687 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:38.520139 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:38.520144 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:38.520148 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:38.520640 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:38.520959 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:38.521363 (+ 404us) server_negotiation.cc:299] Negotiation successful
0807 18:17:38.521666 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":377,"threads_started":1}
I20260807 18:17:38.523998 3632 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:38.619168 3640 heartbeater.cc:503] Master 127.17.110.62:40945 was elected leader, sending a full tablet report...
I20260807 18:17:38.621137 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 3423
W20260807 18:17:38.648636 3059 connection.cc:535] server connection from 127.17.110.2:53021 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 18:17:38.649005 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 3650
Aug 07 18:17:38 dist-test-slave-39l8 krb5kdc[2385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.3@KRBTEST.COM
I20260807 18:17:38.667851 3858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:38.650224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51776 (local address 127.17.110.3:41459)
0807 18:17:38.650938 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:38.650946 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:38.650977 (+ 31us) server_negotiation.cc:407] Connection header received
0807 18:17:38.651040 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:38.651047 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:38.651291 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:38.651473 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:38.652912 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.653932 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.655479 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.656085 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.656297 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:38.661387 (+ 5090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:38.661405 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:38.661423 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:38.661467 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:38.664677 (+ 3210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:38.665492 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:38.665500 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:38.665502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:38.665654 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:38.666071 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:38.666077 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:38.666082 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:38.666480 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:38.666844 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:38.667176 (+ 332us) server_negotiation.cc:299] Negotiation successful
0807 18:17:38.667479 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":332,"threads_started":1}
I20260807 18:17:38.669344 3834 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:38.754060 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 3650
I20260807 18:17:38.779930 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 3035
I20260807 18:17:38.781502 3123 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:38.918107 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 3035
I20260807 18:17:38.960386 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:40945
--webserver_interface=127.17.110.62
--webserver_port=45701
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:37857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:40945
--ranger_config_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.110.20:43015/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.110.20:43015/kms with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:17:39.313318 3866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:39.313668 3866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:39.313766 3866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:39.321378 3866 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:17:39.321527 3866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:39.321573 3866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:39.321624 3866 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:17:39.321674 3866 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:17:39.332413 3866 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:37857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.110.20:43015/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:40945
--ranger_config_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:40945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=45701
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:17:39.333770 3866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:39.335163 3866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:39.348240 3880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:39.348345 3878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:39.348259 3882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:39.349344 3866 server_base.cc:1034] running on GCE node
I20260807 18:17:39.350260 3866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:39.352028 3866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:39.353259 3866 hybrid_clock.cc:648] HybridClock initialized: now 1786126659353225 us; error 58 us; skew 500 ppm
I20260807 18:17:39.353698 3866 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:39 dist-test-slave-39l8 krb5kdc[2385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126659, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:39.357261 3866 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:17:39.359100 3866 webserver.cc:460] Webserver started at http://127.17.110.62:45701/ using document root <none> and password file <none>
I20260807 18:17:39.359813 3866 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:39.359936 3866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 18:17:39 dist-test-slave-39l8 krb5kdc[2385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126659, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for HTTP/127.17.110.20@KRBTEST.COM
I20260807 18:17:39.410890 3866 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 18:17:39.414502 3894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:39.416241 3866 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:17:39.416463 3866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "5598f28d1d0343d19ab15850f27a752c"
format_stamp: "Formatted at 2026-08-07 18:17:34 on dist-test-slave-39l8"
server_key: "f9ba3fdb4dfe5629f357bb5808e4c815"
server_key_iv: "9d8824a3d80bc9e48d93e9a61bc020ab"
server_key_version: "kuduclusterkey@0"
I20260807 18:17:39.417490 3866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:39.430434 3866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:39.435096 3866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:39.435415 3866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:39.459254 3866 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:40945
I20260807 18:17:39.459271 3963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:40945 every 8 connection(s)
I20260807 18:17:39.460959 3866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:17:39.471288 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 3866
I20260807 18:17:39.471594 3964 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:17:39.472272 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.110.20:43015/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.1:36337
--local_ip_for_outbound_sockets=127.17.110.1
--tserver_master_addrs=127.17.110.62:40945
--webserver_port=40275
--webserver_interface=127.17.110.1
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:37857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.110.20:43015/kms with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:17:39.476861 3964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c: Bootstrap starting.
I20260807 18:17:39.505362 3964 log.cc:824] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:39.517967 3964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:17:39.518793 3964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c: Bootstrap complete.
I20260807 18:17:39.532932 3964 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5598f28d1d0343d19ab15850f27a752c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40945 } }
I20260807 18:17:39.534365 3964 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5598f28d1d0343d19ab15850f27a752c, State: Initialized, Role: FOLLOWER
I20260807 18:17:39.535297 3964 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5598f28d1d0343d19ab15850f27a752c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40945 } }
I20260807 18:17:39.535645 3964 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:17:39.535789 3964 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:17:39.535977 3964 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:17:39.542006 3964 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5598f28d1d0343d19ab15850f27a752c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40945 } }
I20260807 18:17:39.542672 3964 leader_election.cc:302] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5598f28d1d0343d19ab15850f27a752c; no voters:
I20260807 18:17:39.543438 3964 leader_election.cc:288] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 18:17:39.543639 3970 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [term 2 FOLLOWER]: Leader election won for term 2
I20260807 18:17:39.552208 3964 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:17:39.554005 3970 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [term 2 LEADER]: Becoming Leader. State: Replica: 5598f28d1d0343d19ab15850f27a752c, State: Running, Role: LEADER
I20260807 18:17:39.554960 3970 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5598f28d1d0343d19ab15850f27a752c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40945 } }
I20260807 18:17:39.564704 3971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5598f28d1d0343d19ab15850f27a752c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5598f28d1d0343d19ab15850f27a752c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40945 } } }
I20260807 18:17:39.565243 3971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [sys.catalog]: This master's current role is: LEADER
I20260807 18:17:39.566210 3983 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:17:39.566162 3974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5598f28d1d0343d19ab15850f27a752c. Latest consensus state: current_term: 2 leader_uuid: "5598f28d1d0343d19ab15850f27a752c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5598f28d1d0343d19ab15850f27a752c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40945 } } }
I20260807 18:17:39.566527 3974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c [sys.catalog]: This master's current role is: LEADER
I20260807 18:17:39.573863 3983 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:17:39.575948 3983 catalog_manager.cc:1259] Loaded cluster ID: 921f81dc1f8840c9ba853f2022ec8f0f
I20260807 18:17:39.576196 3983 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:17:39.581735 3983 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:17:39.584121 3983 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c: Loaded TSK: 0
I20260807 18:17:39.586545 3983 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:17:39.592650 3964 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260807 18:17:40.007612 3967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:40.008268 3967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:40.008548 3967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:40.025141 3967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:40.025463 3967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:40.025785 3967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.1
I20260807 18:17:40.048362 3967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:37857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.110.20:43015/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.1:36337
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.110.1
--webserver_port=40275
--tserver_master_addrs=127.17.110.62:40945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.1
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:17:40.050800 3967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:40.053084 3967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:40.100899 4017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:40.102524 4019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:40.119086 3967 server_base.cc:1034] running on GCE node
W20260807 18:17:40.120393 4025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:40.128273 3967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:40.147123 3967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:40.148504 3967 hybrid_clock.cc:648] HybridClock initialized: now 1786126660148412 us; error 93 us; skew 500 ppm
I20260807 18:17:40.149081 3967 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:40 dist-test-slave-39l8 krb5kdc[2385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126660, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:40.158820 3967 init.cc:377] Logged in from keytab as kudu/127.17.110.1@KRBTEST.COM (short username kudu)
I20260807 18:17:40.186830 3967 webserver.cc:460] Webserver started at http://127.17.110.1:40275/ using document root <none> and password file <none>
I20260807 18:17:40.188118 3967 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:40.188441 3967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 18:17:40 dist-test-slave-39l8 krb5kdc[2385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126660, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for HTTP/127.17.110.20@KRBTEST.COM
I20260807 18:17:40.237663 3967 fs_manager.cc:709] Time spent opening directory manager: real 0.013s user 0.005s sys 0.001s
I20260807 18:17:40.250375 4039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:40.253142 3967 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.007s sys 0.005s
I20260807 18:17:40.253650 3967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-0/wal
uuid: "5fea522d5d5a4017b703d7e35d9a4f7f"
format_stamp: "Formatted at 2026-08-07 18:17:36 on dist-test-slave-39l8"
server_key: "f28283c4a86f00397555772f798b4288"
server_key_iv: "640e6413049891b01a2917a3ac6519eb"
server_key_version: "kuduclusterkey@0"
I20260807 18:17:40.255087 3967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:40.301198 3967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:40.307654 3967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:40.308107 3967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:40.309551 3967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:17:40.311669 3967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:17:40.311810 3967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:40.311956 3967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:17:40.312088 3967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:40.384879 3967 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.1:36337
I20260807 18:17:40.384953 4201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.1:36337 every 8 connection(s)
I20260807 18:17:40.386971 3967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
I20260807 18:17:40.394315 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 3967
I20260807 18:17:40.395372 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.110.20:43015/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.2:33205
--local_ip_for_outbound_sockets=127.17.110.2
--tserver_master_addrs=127.17.110.62:40945
--webserver_port=42759
--webserver_interface=127.17.110.2
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:37857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.110.20:43015/kms with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:17:40 dist-test-slave-39l8 krb5kdc[2385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126660, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:17:40.438373 4206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:40.390824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:41575 (local address 127.17.110.62:40945)
0807 18:17:40.391852 (+ 1028us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:40.391882 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:40.392854 (+ 972us) server_negotiation.cc:407] Connection header received
0807 18:17:40.396370 (+ 3516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:40.396393 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:40.396968 (+ 575us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:40.397499 (+ 531us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:40.400680 (+ 3181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:40.403457 (+ 2777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:40.406726 (+ 3269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:40.408061 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:40.408293 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:40.424436 (+ 16143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:40.424454 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:40.424485 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:40.424565 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:40.430431 (+ 5866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:40.432147 (+ 1716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:40.432160 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:40.432172 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:40.432507 (+ 335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:40.433322 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:40.433331 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:40.433337 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:40.434358 (+ 1021us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:40.434615 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:40.436200 (+ 1585us) server_negotiation.cc:299] Negotiation successful
0807 18:17:40.436768 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":366,"threads_started":1}
I20260807 18:17:40.441771 4207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:40.390943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40945 (local address 127.17.110.1:41575)
0807 18:17:40.392601 (+ 1658us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:40.392654 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:40.395444 (+ 2790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:40.398342 (+ 2898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:40.398354 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:40.399156 (+ 802us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:40.400299 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:40.400339 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:40.403868 (+ 3529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:40.403876 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:40.406399 (+ 2523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:40.406427 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:40.408899 (+ 2472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:40.408911 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:40.409281 (+ 370us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:40.416235 (+ 6954us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:17:40.416284 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:40.423854 (+ 7570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:40.430840 (+ 6986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:40.430855 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:40.430880 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:40.431841 (+ 961us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:40.432770 (+ 929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:40.432779 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:40.432784 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:40.433125 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:40.434808 (+ 1683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:40.434820 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:40.435419 (+ 599us) client_negotiation.cc:769] Sending connection context
0807 18:17:40.439494 (+ 4075us) client_negotiation.cc:240] Negotiation successful
0807 18:17:40.440237 (+ 743us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1344,"thread_start_us":1114,"threads_started":1}
I20260807 18:17:40.446502 4203 heartbeater.cc:347] Connected to a master server at 127.17.110.62:40945
I20260807 18:17:40.446947 4203 heartbeater.cc:464] Registering TS with master...
I20260807 18:17:40.447983 4203 heartbeater.cc:511] Master 127.17.110.62:40945 requested a full tablet report, sending...
W20260807 18:17:40.452591 4203 heartbeater.cc:644] Failed to heartbeat to 127.17.110.62:40945 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 18:17:40.452905 4203 heartbeater.cc:464] Registering TS with master...
I20260807 18:17:40.453269 4203 heartbeater.cc:511] Master 127.17.110.62:40945 requested a full tablet report, sending...
I20260807 18:17:40.455135 4203 heartbeater.cc:464] Registering TS with master...
I20260807 18:17:40.455627 4203 heartbeater.cc:511] Master 127.17.110.62:40945 requested a full tablet report, sending...
W20260807 18:17:40.457083 4203 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:17:40.850665 4209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:40.851114 4209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:40.851321 4209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:40.861030 4209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:40.861218 4209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:40.861416 4209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.2
I20260807 18:17:40.877399 4209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:37857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.110.20:43015/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.2:33205
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.110.2
--webserver_port=42759
--tserver_master_addrs=127.17.110.62:40945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.2
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:17:40.879550 4209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:40.881469 4209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:17:40.928298 4209 server_base.cc:1034] running on GCE node
W20260807 18:17:40.928349 4232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:40.928385 4231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:40.929580 4234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:40.930593 4209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:40.932462 4209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:40.933806 4209 hybrid_clock.cc:648] HybridClock initialized: now 1786126660933723 us; error 67 us; skew 500 ppm
I20260807 18:17:40.934233 4209 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:40 dist-test-slave-39l8 krb5kdc[2385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126660, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:40.938892 4209 init.cc:377] Logged in from keytab as kudu/127.17.110.2@KRBTEST.COM (short username kudu)
I20260807 18:17:40.941485 4209 webserver.cc:460] Webserver started at http://127.17.110.2:42759/ using document root <none> and password file <none>
I20260807 18:17:40.942488 4209 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:40.942685 4209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 18:17:40 dist-test-slave-39l8 krb5kdc[2385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126660, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for HTTP/127.17.110.20@KRBTEST.COM
I20260807 18:17:40.970466 4209 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.004s sys 0.000s
I20260807 18:17:40.975147 4244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:40.977278 4209 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260807 18:17:40.977592 4209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-1/wal
uuid: "84a010898a41411189392239eefdc0d2"
format_stamp: "Formatted at 2026-08-07 18:17:37 on dist-test-slave-39l8"
server_key: "f8f3e230cda5ccc8316dd3cd3be90acd"
server_key_iv: "338d9e179d49f234a1086b48be5785b0"
server_key_version: "kuduclusterkey@0"
I20260807 18:17:40.979295 4209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 07 18:17:40 dist-test-slave-39l8 krb5kdc[2385](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786126660, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:41.029189 4209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:41.035856 4209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:41.036177 4209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:41.037449 4209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:17:41.039299 4209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:17:41.039417 4209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:41.039530 4209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:17:41.039611 4209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:41.107959 4209 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.2:33205
I20260807 18:17:41.108008 4420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.2:33205 every 8 connection(s)
I20260807 18:17:41.109759 4209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
I20260807 18:17:41.117964 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 4209
I20260807 18:17:41.119735 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.110.20:43015/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.3:41459
--local_ip_for_outbound_sockets=127.17.110.3
--tserver_master_addrs=127.17.110.62:40945
--webserver_port=34549
--webserver_interface=127.17.110.3
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:37857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.110.20:43015/kms with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:17:41 dist-test-slave-39l8 krb5kdc[2385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126660, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:17:41.150466 4427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:41.115563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:54891 (local address 127.17.110.62:40945)
0807 18:17:41.124692 (+ 9129us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:41.124702 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:41.125790 (+ 1088us) server_negotiation.cc:407] Connection header received
0807 18:17:41.128696 (+ 2906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:41.128702 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:41.128806 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:41.128918 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:41.131473 (+ 2555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:41.132719 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:41.135555 (+ 2836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:41.136261 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:41.136388 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:41.141637 (+ 5249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:41.141646 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:41.141652 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:41.141701 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:41.145782 (+ 4081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:41.147089 (+ 1307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:41.147097 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:41.147100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:41.147282 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:41.148043 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:41.148049 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:41.148053 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:41.148447 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:41.148635 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:41.149778 (+ 1143us) server_negotiation.cc:299] Negotiation successful
0807 18:17:41.150056 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8934,"thread_start_us":8780,"threads_started":1}
I20260807 18:17:41.153445 4426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:41.114575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40945 (local address 127.17.110.2:54891)
0807 18:17:41.125171 (+ 10596us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:41.125234 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:41.128062 (+ 2828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:41.129457 (+ 1395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:41.129472 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:41.130194 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:41.131191 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:41.131224 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:41.132952 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:41.132959 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:41.135291 (+ 2332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:41.135315 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:41.136863 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:41.136871 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:41.137197 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:41.138167 (+ 970us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:17:41.138202 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:41.141255 (+ 3053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:41.146034 (+ 4779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:41.146045 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:41.146068 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:41.146896 (+ 828us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:41.147464 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:41.147469 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:41.147472 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:41.147883 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:41.148757 (+ 874us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:41.148766 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:41.149269 (+ 503us) client_negotiation.cc:769] Sending connection context
0807 18:17:41.151151 (+ 1882us) client_negotiation.cc:240] Negotiation successful
0807 18:17:41.151837 (+ 686us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10248,"thread_start_us":9933,"threads_started":1}
I20260807 18:17:41.155920 4424 heartbeater.cc:347] Connected to a master server at 127.17.110.62:40945
I20260807 18:17:41.156297 4424 heartbeater.cc:464] Registering TS with master...
I20260807 18:17:41.157282 4424 heartbeater.cc:511] Master 127.17.110.62:40945 requested a full tablet report, sending...
I20260807 18:17:41.159833 3914 ts_manager.cc:194] Registered new tserver with Master: 84a010898a41411189392239eefdc0d2 (127.17.110.2:33205)
I20260807 18:17:41.162477 3914 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.2@KRBTEST.COM'} at 127.17.110.2:54891
I20260807 18:17:41.457270 4203 heartbeater.cc:464] Registering TS with master...
I20260807 18:17:41.457746 4203 heartbeater.cc:511] Master 127.17.110.62:40945 requested a full tablet report, sending...
I20260807 18:17:41.459152 3914 ts_manager.cc:194] Registered new tserver with Master: 5fea522d5d5a4017b703d7e35d9a4f7f (127.17.110.1:36337)
I20260807 18:17:41.460827 3914 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.1@KRBTEST.COM'} at 127.17.110.1:41575
Aug 07 18:17:41 dist-test-slave-39l8 krb5kdc[2385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126660, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for HTTP/127.17.110.20@KRBTEST.COM
W20260807 18:17:41.681526 4428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:41.682129 4428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:41.682368 4428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:41.695276 4428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:41.695539 4428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:41.695815 4428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.3
I20260807 18:17:41.712803 4428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:37857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.110.20:43015/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.3:41459
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.110.3
--webserver_port=34549
--tserver_master_addrs=127.17.110.62:40945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.3
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:17:41.715179 4428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:41.717320 4428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:41.755826 4436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:41.767172 4440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:41.767637 4428 server_base.cc:1034] running on GCE node
W20260807 18:17:41.768097 4435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:41.769196 4428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:41.777097 4428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:41.780690 4428 hybrid_clock.cc:648] HybridClock initialized: now 1786126661780521 us; error 168 us; skew 500 ppm
I20260807 18:17:41.781333 4428 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:41 dist-test-slave-39l8 krb5kdc[2385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126661, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:41.786761 4428 init.cc:377] Logged in from keytab as kudu/127.17.110.3@KRBTEST.COM (short username kudu)
I20260807 18:17:41.818266 4428 webserver.cc:460] Webserver started at http://127.17.110.3:34549/ using document root <none> and password file <none>
I20260807 18:17:41.823467 4428 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:41.823935 4428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 18:17:41 dist-test-slave-39l8 krb5kdc[2385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126661, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for HTTP/127.17.110.20@KRBTEST.COM
Aug 07 18:17:41 dist-test-slave-39l8 krb5kdc[2385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126660, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for HTTP/127.17.110.20@KRBTEST.COM
I20260807 18:17:41.900193 4428 fs_manager.cc:709] Time spent opening directory manager: real 0.025s user 0.004s sys 0.018s
I20260807 18:17:41.907923 4450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:41.910478 4428 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.004s sys 0.002s
I20260807 18:17:41.910871 4428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-2/wal
uuid: "71b7ee428f154c388f33cd1ade59f13a"
format_stamp: "Formatted at 2026-08-07 18:17:38 on dist-test-slave-39l8"
server_key: "3251a48f0567e0d6596ca2b4b23ad556"
server_key_iv: "35ffb5147823cbbebfa277dc81412b2d"
server_key_version: "kuduclusterkey@0"
I20260807 18:17:41.912595 4428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:41.952104 4428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:41.958400 4428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:41.958900 4428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:41.966763 4428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:17:41.975706 4428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:17:41.975940 4428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 18:17:41.976158 4428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:17:41.976284 4428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:42.138885 4428 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.3:41459
I20260807 18:17:42.138939 4615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.3:41459 every 8 connection(s)
I20260807 18:17:42.140755 4428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
I20260807 18:17:42.147742 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 4428
I20260807 18:17:42.168557 4424 heartbeater.cc:503] Master 127.17.110.62:40945 was elected leader, sending a full tablet report...
I20260807 18:17:42.188972 4624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.149118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33748 (local address 127.17.110.62:40945)
0807 18:17:42.167277 (+ 18159us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:42.167285 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:42.167315 (+ 30us) server_negotiation.cc:407] Connection header received
0807 18:17:42.167399 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:42.167406 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:42.167505 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:42.167691 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:42.172346 (+ 4655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.173691 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.176169 (+ 2478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.176919 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.177100 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:42.180949 (+ 3849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:42.180958 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:42.180964 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:42.181020 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:42.185116 (+ 4096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:42.186188 (+ 1072us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:42.186194 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:42.186198 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:42.186380 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:42.186997 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:42.187005 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:42.187010 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:42.187447 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:42.187827 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:42.188231 (+ 404us) server_negotiation.cc:299] Negotiation successful
0807 18:17:42.188531 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17971,"thread_start_us":24720,"threads_started":1}
Aug 07 18:17:42 dist-test-slave-39l8 krb5kdc[2385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126661, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:17:42.197278 4623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.146365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:47929 (local address 127.17.110.62:40945)
0807 18:17:42.162776 (+ 16411us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:42.162783 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:42.162810 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:17:42.175731 (+ 12921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:42.175738 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:42.175831 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:42.175917 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:42.180067 (+ 4150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.181596 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.184738 (+ 3142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.185616 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.185776 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:42.190544 (+ 4768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:42.190551 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:42.190556 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:42.190604 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:42.193687 (+ 3083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:42.194570 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:42.194576 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:42.194581 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:42.194782 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:42.195278 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:42.195283 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:42.195287 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:42.195592 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:42.195774 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:42.196592 (+ 818us) server_negotiation.cc:299] Negotiation successful
0807 18:17:42.196847 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16237,"thread_start_us":16319,"threads_started":1}
I20260807 18:17:42.198330 4621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.145406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40945 (local address 127.17.110.3:47929)
0807 18:17:42.160509 (+ 15103us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:42.160603 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:42.171886 (+ 11283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:42.177891 (+ 6005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:42.177905 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:42.178688 (+ 783us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:42.179733 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:42.179768 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.181884 (+ 2116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.181891 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:42.184480 (+ 2589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:42.184500 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.185911 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.185921 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:42.186300 (+ 379us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:42.187273 (+ 973us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:17:42.187312 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:42.190246 (+ 2934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:42.193868 (+ 3622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:42.193879 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:42.193900 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:42.194410 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:42.194907 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:42.194912 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:42.194916 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:42.195138 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:42.195749 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:42.195759 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:42.196176 (+ 417us) client_negotiation.cc:769] Sending connection context
0807 18:17:42.196478 (+ 302us) client_negotiation.cc:240] Negotiation successful
0807 18:17:42.197130 (+ 652us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14672,"thread_start_us":14314,"threads_started":1}
I20260807 18:17:42.200124 4616 heartbeater.cc:347] Connected to a master server at 127.17.110.62:40945
I20260807 18:17:42.200570 4616 heartbeater.cc:464] Registering TS with master...
I20260807 18:17:42.201502 4616 heartbeater.cc:511] Master 127.17.110.62:40945 requested a full tablet report, sending...
I20260807 18:17:42.203480 3914 ts_manager.cc:194] Registered new tserver with Master: 71b7ee428f154c388f33cd1ade59f13a (127.17.110.3:41459)
I20260807 18:17:42.204598 3914 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.3@KRBTEST.COM'} at 127.17.110.3:47929
I20260807 18:17:42.205402 17848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:17:42.225576 4623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.209393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33756 (local address 127.17.110.62:40945)
0807 18:17:42.209611 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:42.209619 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:42.210117 (+ 498us) server_negotiation.cc:407] Connection header received
0807 18:17:42.210319 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:42.210324 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:42.210413 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:42.210522 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:42.212473 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.213302 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.214961 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.215464 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.215594 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:42.218682 (+ 3088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:42.218688 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:42.218692 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:42.218843 (+ 151us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:42.222212 (+ 3369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:42.223242 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:42.223248 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:42.223251 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:42.223407 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:42.223836 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:42.223841 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:42.223843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:42.224237 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:42.224452 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:42.224911 (+ 459us) server_negotiation.cc:299] Negotiation successful
0807 18:17:42.225149 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 18:17:42.231652 3914 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33756:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:17:42.240476 3914 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:17:42.277503 4654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.259518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:33205 (local address 127.0.0.1:49370)
0807 18:17:42.260667 (+ 1149us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:42.260691 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:42.260833 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:42.264814 (+ 3981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:42.264826 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:42.264874 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:42.267235 (+ 2361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:42.267257 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.270928 (+ 3671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.270934 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:42.273858 (+ 2924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:42.273876 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.276393 (+ 2517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.276399 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:42.276628 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:42.276685 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 18:17:42.276848 (+ 163us) client_negotiation.cc:240] Negotiation successful
0807 18:17:42.277062 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":988,"thread_start_us":843,"threads_started":1}
I20260807 18:17:42.277588 4653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.259234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49370 (local address 127.17.110.2:33205)
0807 18:17:42.263555 (+ 4321us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:42.263569 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:42.263612 (+ 43us) server_negotiation.cc:407] Connection header received
0807 18:17:42.263757 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:42.263771 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:42.264136 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:42.264418 (+ 282us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:42.267617 (+ 3199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.270669 (+ 3052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.274086 (+ 3417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.275785 (+ 1699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.275929 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:42.276195 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:42.276940 (+ 745us) server_negotiation.cc:299] Negotiation successful
0807 18:17:42.277158 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4078,"thread_start_us":3350,"threads_started":1}
I20260807 18:17:42.278203 4648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.258234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:36337 (local address 127.0.0.1:47652)
0807 18:17:42.259625 (+ 1391us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:42.259678 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:42.259968 (+ 290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:42.270385 (+ 10417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:42.270389 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:42.270420 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:42.270924 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:42.270938 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.273206 (+ 2268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.273212 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:42.275357 (+ 2145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:42.275372 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.277372 (+ 2000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.277377 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:42.277565 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:42.277605 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 18:17:42.277682 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 18:17:42.277826 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1066,"mutex_wait_us":51,"thread_start_us":864,"threads_started":1}
I20260807 18:17:42.279155 4651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.258423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47652 (local address 127.17.110.1:36337)
0807 18:17:42.269588 (+ 11165us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:42.269595 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:42.269623 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:17:42.269789 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:42.269796 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:42.270114 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:42.271166 (+ 1052us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:42.271822 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.273049 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.275593 (+ 2544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.277118 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.277964 (+ 846us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:42.278249 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:42.278359 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 18:17:42.278726 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11001,"thread_start_us":11141,"threads_started":1}
I20260807 18:17:42.279781 4655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.260431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:41459 (local address 127.0.0.1:34822)
0807 18:17:42.270290 (+ 9859us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:42.270314 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:42.270467 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:42.271503 (+ 1036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:42.271511 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:42.271537 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:42.272042 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:42.272060 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.274263 (+ 2203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.274271 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:42.276413 (+ 2142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:42.276432 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.278910 (+ 2478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.278917 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:42.279090 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:42.279121 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 18:17:42.279222 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 18:17:42.279361 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9688,"mutex_wait_us":1,"thread_start_us":10138,"threads_started":1}
I20260807 18:17:42.280166 4656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.261221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34822 (local address 127.17.110.3:41459)
0807 18:17:42.269510 (+ 8289us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:42.269527 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:42.270829 (+ 1302us) server_negotiation.cc:407] Connection header received
0807 18:17:42.270953 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:42.270964 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:42.271270 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:42.271491 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:42.272318 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.274108 (+ 1790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.276659 (+ 2551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.278393 (+ 1734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.278551 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:42.278843 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:42.279377 (+ 534us) server_negotiation.cc:299] Negotiation successful
0807 18:17:42.279579 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8000,"thread_start_us":7768,"threads_started":1}
I20260807 18:17:42.282465 4322 tablet_service.cc:1467] Processing CreateTablet for tablet 0f53d7406956401985600be3aac142dc (DEFAULT_TABLE table=test-table [id=b0e0381c79014f8db14fb87d6605eb55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:17:42.283412 4106 tablet_service.cc:1467] Processing CreateTablet for tablet 0f53d7406956401985600be3aac142dc (DEFAULT_TABLE table=test-table [id=b0e0381c79014f8db14fb87d6605eb55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:17:42.284168 4322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f53d7406956401985600be3aac142dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:42.283811 4527 tablet_service.cc:1467] Processing CreateTablet for tablet 0f53d7406956401985600be3aac142dc (DEFAULT_TABLE table=test-table [id=b0e0381c79014f8db14fb87d6605eb55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:17:42.284749 4106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f53d7406956401985600be3aac142dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:42.285219 4527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f53d7406956401985600be3aac142dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:42.301234 4657 tablet_bootstrap.cc:492] T 0f53d7406956401985600be3aac142dc P 5fea522d5d5a4017b703d7e35d9a4f7f: Bootstrap starting.
I20260807 18:17:42.303702 4658 tablet_bootstrap.cc:492] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2: Bootstrap starting.
I20260807 18:17:42.304659 4659 tablet_bootstrap.cc:492] T 0f53d7406956401985600be3aac142dc P 71b7ee428f154c388f33cd1ade59f13a: Bootstrap starting.
I20260807 18:17:42.306015 4657 tablet_bootstrap.cc:654] T 0f53d7406956401985600be3aac142dc P 5fea522d5d5a4017b703d7e35d9a4f7f: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:42.307667 4657 log.cc:824] T 0f53d7406956401985600be3aac142dc P 5fea522d5d5a4017b703d7e35d9a4f7f: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:42.308284 4658 tablet_bootstrap.cc:654] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:42.309355 4659 tablet_bootstrap.cc:654] T 0f53d7406956401985600be3aac142dc P 71b7ee428f154c388f33cd1ade59f13a: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:42.309373 4658 log.cc:824] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:42.311233 4659 log.cc:824] T 0f53d7406956401985600be3aac142dc P 71b7ee428f154c388f33cd1ade59f13a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:42.315202 4659 tablet_bootstrap.cc:492] T 0f53d7406956401985600be3aac142dc P 71b7ee428f154c388f33cd1ade59f13a: No bootstrap required, opened a new log
I20260807 18:17:42.315526 4658 tablet_bootstrap.cc:492] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2: No bootstrap required, opened a new log
I20260807 18:17:42.315549 4659 ts_tablet_manager.cc:1397] T 0f53d7406956401985600be3aac142dc P 71b7ee428f154c388f33cd1ade59f13a: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260807 18:17:42.315828 4658 ts_tablet_manager.cc:1397] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
I20260807 18:17:42.315883 4657 tablet_bootstrap.cc:492] T 0f53d7406956401985600be3aac142dc P 5fea522d5d5a4017b703d7e35d9a4f7f: No bootstrap required, opened a new log
I20260807 18:17:42.316157 4657 ts_tablet_manager.cc:1397] T 0f53d7406956401985600be3aac142dc P 5fea522d5d5a4017b703d7e35d9a4f7f: Time spent bootstrapping tablet: real 0.015s user 0.012s sys 0.000s
I20260807 18:17:42.325101 4659 raft_consensus.cc:348] T 0f53d7406956401985600be3aac142dc P 71b7ee428f154c388f33cd1ade59f13a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fea522d5d5a4017b703d7e35d9a4f7f" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 36337 } } peers { permanent_uuid: "84a010898a41411189392239eefdc0d2" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 33205 } } peers { permanent_uuid: "71b7ee428f154c388f33cd1ade59f13a" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 41459 } }
I20260807 18:17:42.325664 4659 raft_consensus.cc:374] T 0f53d7406956401985600be3aac142dc P 71b7ee428f154c388f33cd1ade59f13a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:42.325784 4659 raft_consensus.cc:729] T 0f53d7406956401985600be3aac142dc P 71b7ee428f154c388f33cd1ade59f13a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71b7ee428f154c388f33cd1ade59f13a, State: Initialized, Role: FOLLOWER
I20260807 18:17:42.326534 4659 consensus_queue.cc:260] T 0f53d7406956401985600be3aac142dc P 71b7ee428f154c388f33cd1ade59f13a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fea522d5d5a4017b703d7e35d9a4f7f" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 36337 } } peers { permanent_uuid: "84a010898a41411189392239eefdc0d2" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 33205 } } peers { permanent_uuid: "71b7ee428f154c388f33cd1ade59f13a" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 41459 } }
I20260807 18:17:42.327447 4658 raft_consensus.cc:348] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fea522d5d5a4017b703d7e35d9a4f7f" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 36337 } } peers { permanent_uuid: "84a010898a41411189392239eefdc0d2" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 33205 } } peers { permanent_uuid: "71b7ee428f154c388f33cd1ade59f13a" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 41459 } }
I20260807 18:17:42.327447 4657 raft_consensus.cc:348] T 0f53d7406956401985600be3aac142dc P 5fea522d5d5a4017b703d7e35d9a4f7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fea522d5d5a4017b703d7e35d9a4f7f" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 36337 } } peers { permanent_uuid: "84a010898a41411189392239eefdc0d2" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 33205 } } peers { permanent_uuid: "71b7ee428f154c388f33cd1ade59f13a" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 41459 } }
I20260807 18:17:42.327909 4616 heartbeater.cc:503] Master 127.17.110.62:40945 was elected leader, sending a full tablet report...
I20260807 18:17:42.327925 4657 raft_consensus.cc:374] T 0f53d7406956401985600be3aac142dc P 5fea522d5d5a4017b703d7e35d9a4f7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:42.328042 4657 raft_consensus.cc:729] T 0f53d7406956401985600be3aac142dc P 5fea522d5d5a4017b703d7e35d9a4f7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fea522d5d5a4017b703d7e35d9a4f7f, State: Initialized, Role: FOLLOWER
I20260807 18:17:42.327998 4658 raft_consensus.cc:374] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:42.328199 4658 raft_consensus.cc:729] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84a010898a41411189392239eefdc0d2, State: Initialized, Role: FOLLOWER
I20260807 18:17:42.328823 4659 ts_tablet_manager.cc:1428] T 0f53d7406956401985600be3aac142dc P 71b7ee428f154c388f33cd1ade59f13a: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
I20260807 18:17:42.328794 4657 consensus_queue.cc:260] T 0f53d7406956401985600be3aac142dc P 5fea522d5d5a4017b703d7e35d9a4f7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fea522d5d5a4017b703d7e35d9a4f7f" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 36337 } } peers { permanent_uuid: "84a010898a41411189392239eefdc0d2" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 33205 } } peers { permanent_uuid: "71b7ee428f154c388f33cd1ade59f13a" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 41459 } }
I20260807 18:17:42.329063 4658 consensus_queue.cc:260] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fea522d5d5a4017b703d7e35d9a4f7f" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 36337 } } peers { permanent_uuid: "84a010898a41411189392239eefdc0d2" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 33205 } } peers { permanent_uuid: "71b7ee428f154c388f33cd1ade59f13a" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 41459 } }
I20260807 18:17:42.330196 4203 heartbeater.cc:503] Master 127.17.110.62:40945 was elected leader, sending a full tablet report...
I20260807 18:17:42.331343 4657 ts_tablet_manager.cc:1428] T 0f53d7406956401985600be3aac142dc P 5fea522d5d5a4017b703d7e35d9a4f7f: Time spent starting tablet: real 0.015s user 0.015s sys 0.001s
I20260807 18:17:42.333524 4658 ts_tablet_manager.cc:1428] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2: Time spent starting tablet: real 0.018s user 0.017s sys 0.000s
W20260807 18:17:42.368693 4425 tablet.cc:2369] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:17:42.390377 4205 tablet.cc:2369] T 0f53d7406956401985600be3aac142dc P 5fea522d5d5a4017b703d7e35d9a4f7f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:17:42.401574 4666 raft_consensus.cc:482] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:17:42.401930 4666 raft_consensus.cc:504] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fea522d5d5a4017b703d7e35d9a4f7f" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 36337 } } peers { permanent_uuid: "84a010898a41411189392239eefdc0d2" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 33205 } } peers { permanent_uuid: "71b7ee428f154c388f33cd1ade59f13a" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 41459 } }
I20260807 18:17:42.403903 4666 leader_election.cc:288] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5fea522d5d5a4017b703d7e35d9a4f7f (127.17.110.1:36337), 71b7ee428f154c388f33cd1ade59f13a (127.17.110.3:41459)
I20260807 18:17:42.411911 4651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.404632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:44951 (local address 127.17.110.1:36337)
0807 18:17:42.404920 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:42.404928 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:42.405279 (+ 351us) server_negotiation.cc:407] Connection header received
0807 18:17:42.405593 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:42.405603 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:42.405721 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:42.405806 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:42.406926 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.407744 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.409902 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.410698 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.410904 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:42.411014 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:42.411499 (+ 485us) server_negotiation.cc:299] Negotiation successful
0807 18:17:42.411641 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 18:17:42.411947 4656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.405377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:37451 (local address 127.17.110.3:41459)
0807 18:17:42.405577 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:42.405583 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:42.406041 (+ 458us) server_negotiation.cc:407] Connection header received
0807 18:17:42.406158 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:42.406166 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:42.406259 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:42.406370 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:42.407165 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.408060 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.409893 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.410844 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.411027 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:42.411110 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:42.411454 (+ 344us) server_negotiation.cc:299] Negotiation successful
0807 18:17:42.411600 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 18:17:42.412061 4668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.405214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:41459 (local address 127.17.110.2:37451)
0807 18:17:42.405915 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:42.405931 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:42.406048 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:42.406413 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:42.406418 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:42.406447 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:42.406978 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:42.406996 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.408225 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.408229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:42.409710 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:42.409723 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.411101 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.411116 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:42.411272 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:42.411306 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 18:17:42.411386 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 18:17:42.411550 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"mutex_wait_us":10,"thread_start_us":376,"threads_started":1}
I20260807 18:17:42.412125 4667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.404364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:36337 (local address 127.17.110.2:44951)
0807 18:17:42.405128 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:42.405160 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:42.405350 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:42.406068 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:42.406076 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:42.406112 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:42.406669 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:42.406686 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.407894 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.407900 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:42.409713 (+ 1813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:42.409727 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.411010 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.411018 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:42.411297 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:42.411324 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 18:17:42.411451 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 18:17:42.411600 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"mutex_wait_us":89,"thread_start_us":369,"threads_started":1}
W20260807 18:17:42.412889 4618 tablet.cc:2369] T 0f53d7406956401985600be3aac142dc P 71b7ee428f154c388f33cd1ade59f13a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:17:42.413261 4136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f53d7406956401985600be3aac142dc" candidate_uuid: "84a010898a41411189392239eefdc0d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fea522d5d5a4017b703d7e35d9a4f7f" is_pre_election: true
I20260807 18:17:42.413241 4558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f53d7406956401985600be3aac142dc" candidate_uuid: "84a010898a41411189392239eefdc0d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71b7ee428f154c388f33cd1ade59f13a" is_pre_election: true
I20260807 18:17:42.413815 4136 raft_consensus.cc:2454] T 0f53d7406956401985600be3aac142dc P 5fea522d5d5a4017b703d7e35d9a4f7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84a010898a41411189392239eefdc0d2 in term 0.
I20260807 18:17:42.413861 4558 raft_consensus.cc:2454] T 0f53d7406956401985600be3aac142dc P 71b7ee428f154c388f33cd1ade59f13a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84a010898a41411189392239eefdc0d2 in term 0.
I20260807 18:17:42.414619 4247 leader_election.cc:302] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fea522d5d5a4017b703d7e35d9a4f7f, 84a010898a41411189392239eefdc0d2; no voters:
I20260807 18:17:42.415251 4666 raft_consensus.cc:2781] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:17:42.415441 4666 raft_consensus.cc:482] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:17:42.416018 4666 raft_consensus.cc:3037] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:42.418942 4666 raft_consensus.cc:504] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fea522d5d5a4017b703d7e35d9a4f7f" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 36337 } } peers { permanent_uuid: "84a010898a41411189392239eefdc0d2" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 33205 } } peers { permanent_uuid: "71b7ee428f154c388f33cd1ade59f13a" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 41459 } }
I20260807 18:17:42.419734 4666 leader_election.cc:288] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2 [CANDIDATE]: Term 1 election: Requested vote from peers 5fea522d5d5a4017b703d7e35d9a4f7f (127.17.110.1:36337), 71b7ee428f154c388f33cd1ade59f13a (127.17.110.3:41459)
I20260807 18:17:42.420466 4136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f53d7406956401985600be3aac142dc" candidate_uuid: "84a010898a41411189392239eefdc0d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fea522d5d5a4017b703d7e35d9a4f7f"
I20260807 18:17:42.420521 4558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f53d7406956401985600be3aac142dc" candidate_uuid: "84a010898a41411189392239eefdc0d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71b7ee428f154c388f33cd1ade59f13a"
I20260807 18:17:42.420724 4136 raft_consensus.cc:3037] T 0f53d7406956401985600be3aac142dc P 5fea522d5d5a4017b703d7e35d9a4f7f [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:42.420812 4558 raft_consensus.cc:3037] T 0f53d7406956401985600be3aac142dc P 71b7ee428f154c388f33cd1ade59f13a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:42.423731 4136 raft_consensus.cc:2454] T 0f53d7406956401985600be3aac142dc P 5fea522d5d5a4017b703d7e35d9a4f7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84a010898a41411189392239eefdc0d2 in term 1.
I20260807 18:17:42.423755 4558 raft_consensus.cc:2454] T 0f53d7406956401985600be3aac142dc P 71b7ee428f154c388f33cd1ade59f13a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84a010898a41411189392239eefdc0d2 in term 1.
I20260807 18:17:42.424363 4247 leader_election.cc:302] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71b7ee428f154c388f33cd1ade59f13a, 84a010898a41411189392239eefdc0d2; no voters:
I20260807 18:17:42.424770 4666 raft_consensus.cc:2781] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:17:42.425258 4666 raft_consensus.cc:686] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2 [term 1 LEADER]: Becoming Leader. State: Replica: 84a010898a41411189392239eefdc0d2, State: Running, Role: LEADER
I20260807 18:17:42.425818 4666 consensus_queue.cc:237] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fea522d5d5a4017b703d7e35d9a4f7f" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 36337 } } peers { permanent_uuid: "84a010898a41411189392239eefdc0d2" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 33205 } } peers { permanent_uuid: "71b7ee428f154c388f33cd1ade59f13a" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 41459 } }
I20260807 18:17:42.432211 3909 catalog_manager.cc:5495] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84a010898a41411189392239eefdc0d2 (127.17.110.2). New cstate: current_term: 1 leader_uuid: "84a010898a41411189392239eefdc0d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fea522d5d5a4017b703d7e35d9a4f7f" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 36337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84a010898a41411189392239eefdc0d2" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 33205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "71b7ee428f154c388f33cd1ade59f13a" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 41459 } health_report { overall_health: UNKNOWN } } }
I20260807 18:17:42.467386 4653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.461553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49382 (local address 127.17.110.2:33205)
0807 18:17:42.461760 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:42.461768 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:42.461827 (+ 59us) server_negotiation.cc:407] Connection header received
0807 18:17:42.462173 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:42.462178 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:42.462263 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:42.462436 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:17:42.462902 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.463660 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.465072 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.465484 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.465626 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:42.466181 (+ 555us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:17:42.466776 (+ 595us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:17:42.466952 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:42.467016 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 18:17:42.467115 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:17:42.476294 4558 raft_consensus.cc:1261] T 0f53d7406956401985600be3aac142dc P 71b7ee428f154c388f33cd1ade59f13a [term 1 FOLLOWER]: Refusing update from remote peer 84a010898a41411189392239eefdc0d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:17:42.476415 4136 raft_consensus.cc:1261] T 0f53d7406956401985600be3aac142dc P 5fea522d5d5a4017b703d7e35d9a4f7f [term 1 FOLLOWER]: Refusing update from remote peer 84a010898a41411189392239eefdc0d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:17:42.477578 4669 consensus_queue.cc:1035] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71b7ee428f154c388f33cd1ade59f13a" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 41459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:17:42.478004 4666 consensus_queue.cc:1035] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fea522d5d5a4017b703d7e35d9a4f7f" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 36337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:17:42.491019 4674 mvcc.cc:205] Tried to move back new op lower bound from 7315974809491664896 to 7315974809300889600. Current Snapshot: MvccSnapshot[applied={T|T < 7315974809491664896}]
I20260807 18:17:42.505896 3909 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:17:42.506299 3909 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:17:42.510736 3909 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5598f28d1d0343d19ab15850f27a752c: Sending DeleteTablet for 3 replicas of tablet 0f53d7406956401985600be3aac142dc
I20260807 18:17:42.512265 4322 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f53d7406956401985600be3aac142dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:17:42 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:49370
I20260807 18:17:42.512254 4527 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f53d7406956401985600be3aac142dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:17:42 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:34822
I20260807 18:17:42.518069 4106 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f53d7406956401985600be3aac142dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:17:42 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:47652
I20260807 18:17:42.522068 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 3967
I20260807 18:17:42.527870 4687 tablet_replica.cc:330] T 0f53d7406956401985600be3aac142dc P 71b7ee428f154c388f33cd1ade59f13a: stopping tablet replica
I20260807 18:17:42.528638 4689 tablet_replica.cc:330] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2: stopping tablet replica
I20260807 18:17:42.529177 4689 raft_consensus.cc:2229] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:17:42.529270 4687 raft_consensus.cc:2229] T 0f53d7406956401985600be3aac142dc P 71b7ee428f154c388f33cd1ade59f13a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:17:42.529855 4687 raft_consensus.cc:2258] T 0f53d7406956401985600be3aac142dc P 71b7ee428f154c388f33cd1ade59f13a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:17:42.529932 4689 raft_consensus.cc:2258] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:17:42.532867 4689 ts_tablet_manager.cc:1905] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:17:42.537113 4690 tablet_replica.cc:330] T 0f53d7406956401985600be3aac142dc P 5fea522d5d5a4017b703d7e35d9a4f7f: stopping tablet replica
I20260807 18:17:42.537622 4690 raft_consensus.cc:2229] T 0f53d7406956401985600be3aac142dc P 5fea522d5d5a4017b703d7e35d9a4f7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:17:42.538273 4690 raft_consensus.cc:2258] T 0f53d7406956401985600be3aac142dc P 5fea522d5d5a4017b703d7e35d9a4f7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:17:42.541131 4690 ts_tablet_manager.cc:1905] T 0f53d7406956401985600be3aac142dc P 5fea522d5d5a4017b703d7e35d9a4f7f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:17:42.542665 4689 ts_tablet_manager.cc:1918] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:17:42.542975 4689 log.cc:1196] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-1/wal/wals/0f53d7406956401985600be3aac142dc
I20260807 18:17:42.543712 4689 ts_tablet_manager.cc:1939] T 0f53d7406956401985600be3aac142dc P 84a010898a41411189392239eefdc0d2: Deleting consensus metadata
I20260807 18:17:42.545349 4687 ts_tablet_manager.cc:1905] T 0f53d7406956401985600be3aac142dc P 71b7ee428f154c388f33cd1ade59f13a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:17:42.545873 4651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.523787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47664 (local address 127.17.110.1:36337)
0807 18:17:42.524228 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:42.524234 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:42.524256 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:17:42.524323 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:42.524329 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:42.524413 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:42.524518 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:42.526634 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.527830 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.531014 (+ 3184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.531774 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.531915 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:42.535910 (+ 3995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:42.535931 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:42.535954 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:42.536018 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:42.541029 (+ 5011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:42.542190 (+ 1161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:42.542200 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:42.542204 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:42.542407 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:42.542988 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:42.542993 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:42.542996 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:42.543498 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:42.543840 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:42.545180 (+ 1340us) server_negotiation.cc:299] Negotiation successful
0807 18:17:42.545482 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274}
I20260807 18:17:42.545935 3898 catalog_manager.cc:4841] TS 84a010898a41411189392239eefdc0d2 (127.17.110.2:33205): tablet 0f53d7406956401985600be3aac142dc (table test-table [id=b0e0381c79014f8db14fb87d6605eb55]) successfully deleted
I20260807 18:17:42.547142 4197 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:42.554342 4687 ts_tablet_manager.cc:1918] T 0f53d7406956401985600be3aac142dc P 71b7ee428f154c388f33cd1ade59f13a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:17:42.554558 4687 log.cc:1196] T 0f53d7406956401985600be3aac142dc P 71b7ee428f154c388f33cd1ade59f13a: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126429568977-17848-0/minicluster-data/ts-2/wal/wals/0f53d7406956401985600be3aac142dc
I20260807 18:17:42.555116 4687 ts_tablet_manager.cc:1939] T 0f53d7406956401985600be3aac142dc P 71b7ee428f154c388f33cd1ade59f13a: Deleting consensus metadata
I20260807 18:17:42.556823 3896 catalog_manager.cc:4841] TS 71b7ee428f154c388f33cd1ade59f13a (127.17.110.3:41459): tablet 0f53d7406956401985600be3aac142dc (table test-table [id=b0e0381c79014f8db14fb87d6605eb55]) successfully deleted
I20260807 18:17:42.672950 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 3967
W20260807 18:17:42.701330 3896 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.110.1:36337 (error 108)
I20260807 18:17:42.701562 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 4209
I20260807 18:17:42.703464 4655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.702955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:36337 (local address 127.0.0.1:47670)
0807 18:17:42.703152 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:42.703279 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:36337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260807 18:17:42.703737 3896 catalog_manager.cc:4584] TS 5fea522d5d5a4017b703d7e35d9a4f7f (127.17.110.1:36337): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0f53d7406956401985600be3aac142dc: Network error: Client connection negotiation failed: client connection to 127.17.110.1:36337: connect: Connection refused (error 111)
I20260807 18:17:42.717249 4653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.702602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49388 (local address 127.17.110.2:33205)
0807 18:17:42.702849 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:42.702855 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:42.702899 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:17:42.702970 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:42.702976 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:42.703104 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:42.703239 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:42.704859 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.705702 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.707064 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.707538 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.707699 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:42.710566 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:42.710583 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:42.710600 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:42.710651 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:42.713963 (+ 3312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:42.714879 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:42.714887 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:42.714889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:42.715061 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:42.715505 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:42.715509 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:42.715511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:42.715906 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:42.716258 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:42.716596 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 18:17:42.716868 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:17:42.718111 4413 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:42.757732 4655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.757164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:36337 (local address 127.0.0.1:47686)
0807 18:17:42.757413 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:42.757530 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:36337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260807 18:17:42.759021 4655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.758616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:36337 (local address 127.0.0.1:47688)
0807 18:17:42.758806 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:42.758891 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:36337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 18:17:42.819864 4655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.819257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:36337 (local address 127.0.0.1:47694)
0807 18:17:42.819574 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:42.819690 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:36337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260807 18:17:42.821058 4655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.820687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:36337 (local address 127.0.0.1:47698)
0807 18:17:42.820845 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:42.820930 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:36337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 18:17:42.838419 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 4209
I20260807 18:17:42.866457 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 4428
I20260807 18:17:42.881788 4656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.867336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34826 (local address 127.17.110.3:41459)
0807 18:17:42.867560 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:42.867568 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:42.867708 (+ 140us) server_negotiation.cc:407] Connection header received
0807 18:17:42.867923 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:42.867935 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:42.868004 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:42.868177 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:42.869573 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.870302 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.871861 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.872284 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.872438 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:42.875512 (+ 3074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:42.875530 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:42.875547 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:42.875606 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:42.878607 (+ 3001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:42.879513 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:42.879521 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:42.879524 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:42.879679 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:42.880092 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:42.880099 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:42.880102 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:42.880506 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:42.880851 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:42.881167 (+ 316us) server_negotiation.cc:299] Negotiation successful
0807 18:17:42.881411 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 18:17:42.882521 4608 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:42.902628 4655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.901965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:36337 (local address 127.0.0.1:47710)
0807 18:17:42.902256 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:42.902384 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:36337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260807 18:17:42.903952 4655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.903453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:36337 (local address 127.0.0.1:47716)
0807 18:17:42.903699 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:42.903797 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:36337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
I20260807 18:17:42.977115 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 4428
I20260807 18:17:43.008826 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 3866
I20260807 18:17:43.010380 3954 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:43.136060 4655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:43.135482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:36337 (local address 127.0.0.1:47732)
0807 18:17:43.135739 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:43.135865 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:36337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260807 18:17:43.136229 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 3866
I20260807 18:17:43.181142 17848 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 18:17:43.528563 17848 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 18:17:43.528891 17848 mini_ranger.cc:66] Stopping Ranger...
I20260807 18:17:43.886426 17848 mini_ranger.cc:68] Stopped Ranger
2026-08-07 18:17:43.886 UTC [2427] LOG: received smart shutdown request
2026-08-07 18:17:43.890 UTC [2427] LOG: background worker "logical replication launcher" (PID 2443) exited with exit code 1
2026-08-07 18:17:43.891 UTC [2434] LOG: shutting down
2026-08-07 18:17:44.089 UTC [2427] LOG: database system is shut down
2026-08-07T18:17:44Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (61608 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:17:44 dist-test-slave-39l8 krb5kdc[4714](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:17:44 dist-test-slave-39l8 krb5kdc[4714](info): set up 2 sockets
Aug 07 18:17:44 dist-test-slave-39l8 krb5kdc[4714](info): commencing operation
krb5kdc: starting...
W20260807 18:17:45.546356 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.283s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:17:45 dist-test-slave-39l8 krb5kdc[4714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:17:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:17:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.110.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
I20260807 18:17:45.850663 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:38075
--webserver_interface=127.17.110.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:37991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:38075,127.17.110.61:35657,127.17.110.60:46033
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:17:46.230406 4732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:46.230854 4732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:46.231030 4732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:46.238821 4732 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:17:46.239027 4732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:46.239153 4732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:46.239259 4732 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:17:46.239352 4732 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:17:46.249990 4732 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:37991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:38075,127.17.110.61:35657,127.17.110.60:46033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:38075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:17:46.251606 4732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:46.253047 4732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:46.266745 4740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:46.266794 4744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:46.267099 4741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:46.268055 4732 server_base.cc:1034] running on GCE node
I20260807 18:17:46.268963 4732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:46.270550 4732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:46.271833 4732 hybrid_clock.cc:648] HybridClock initialized: now 1786126666271758 us; error 79 us; skew 500 ppm
I20260807 18:17:46.272331 4732 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:46 dist-test-slave-39l8 krb5kdc[4714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126666, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:46.281319 4732 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:17:46.283398 4732 webserver.cc:460] Webserver started at http://127.17.110.62:36707/ using document root <none> and password file <none>
I20260807 18:17:46.284222 4732 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:46.284391 4732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:46.284754 4732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:46.288352 4732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "5f4a971bf593423592de0955a984311c"
format_stamp: "Formatted at 2026-08-07 18:17:46 on dist-test-slave-39l8"
server_key: "1f147a9509d32ea9993b20b8b08f1670"
server_key_iv: "9edc150fa787fa549d95cbcc83f731e8"
server_key_version: "encryptionkey@0"
I20260807 18:17:46.289435 4732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "5f4a971bf593423592de0955a984311c"
format_stamp: "Formatted at 2026-08-07 18:17:46 on dist-test-slave-39l8"
server_key: "1f147a9509d32ea9993b20b8b08f1670"
server_key_iv: "9edc150fa787fa549d95cbcc83f731e8"
server_key_version: "encryptionkey@0"
I20260807 18:17:46.295643 4732 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260807 18:17:46.299782 4752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:46.301465 4732 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:17:46.301746 4732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "5f4a971bf593423592de0955a984311c"
format_stamp: "Formatted at 2026-08-07 18:17:46 on dist-test-slave-39l8"
server_key: "1f147a9509d32ea9993b20b8b08f1670"
server_key_iv: "9edc150fa787fa549d95cbcc83f731e8"
server_key_version: "encryptionkey@0"
I20260807 18:17:46.302016 4732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:46.323530 4732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:46.328879 4732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:46.329244 4732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:46.351290 4732 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:38075
I20260807 18:17:46.351327 4825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:38075 every 8 connection(s)
I20260807 18:17:46.352981 4732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:17:46.357299 4826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:46.362433 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 4732
I20260807 18:17:46.362784 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:17:46.363535 17848 external_mini_cluster.cc:1428] Setting key 353e50bf23f90483b3110a929aa53c5a
I20260807 18:17:46.365016 4826 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38075 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:17:46 dist-test-slave-39l8 krb5kdc[4714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126666, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.110.61@KRBTEST.COM; defaulting to no policy
I20260807 18:17:46.389736 4829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:46.368711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51316 (local address 127.17.110.62:38075)
0807 18:17:46.369700 (+ 989us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:46.369724 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:46.370223 (+ 499us) server_negotiation.cc:407] Connection header received
0807 18:17:46.372451 (+ 2228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:46.372457 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:46.372713 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:46.372944 (+ 231us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:46.374362 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.375706 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.377202 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.377807 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.378043 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:46.381820 (+ 3777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:46.381829 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:46.381844 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:46.381876 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:46.384975 (+ 3099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:46.385742 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:46.385764 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:46.385766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:46.385925 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:46.386361 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:46.386368 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:46.386371 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:46.387010 (+ 639us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:46.387240 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:46.387798 (+ 558us) server_negotiation.cc:299] Negotiation successful
0807 18:17:46.388182 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":713,"thread_start_us":443,"threads_started":1}
I20260807 18:17:46.389772 4830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:46.368727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38075 (local address 127.0.0.1:51316)
0807 18:17:46.370035 (+ 1308us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:46.370073 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:46.372023 (+ 1950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:46.372968 (+ 945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:46.372974 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:46.373768 (+ 794us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:46.374133 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:46.374152 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.375890 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.375896 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:46.377032 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:46.377047 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.378086 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.378094 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:46.378273 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:46.379180 (+ 907us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:17:46.379206 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:46.381540 (+ 2334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:46.385138 (+ 3598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:46.385150 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:46.385164 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:46.385594 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:46.386050 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:46.386055 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:46.386059 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:46.386237 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:46.387161 (+ 924us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:46.387169 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:46.387398 (+ 229us) client_negotiation.cc:769] Sending connection context
0807 18:17:46.387666 (+ 268us) client_negotiation.cc:240] Negotiation successful
0807 18:17:46.388311 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1094,"mutex_wait_us":152,"thread_start_us":842,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.61@KRBTEST.COM" created.
I20260807 18:17:46.393388 4826 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35657 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:17:46.394954 4830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:46.394350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:35657 (local address 127.0.0.1:56422)
0807 18:17:46.394580 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:46.394771 (+ 191us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.61:35657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260807 18:17:46.395319 4757 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.110.61:35657: connect: Connection refused (error 111)
I20260807 18:17:46.397245 4830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:46.396717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:35657 (local address 127.0.0.1:56434)
0807 18:17:46.396928 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:46.397037 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.61:35657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260807 18:17:46.397497 4826 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.61:35657: Network error: Client connection negotiation failed: client connection to 127.17.110.61:35657: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.61.keytab.
Entry for principal kudu/127.17.110.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.61.keytab.
I20260807 18:17:46.427772 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.61:35657
--webserver_interface=127.17.110.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.61.keytab
--principal=kudu/127.17.110.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:37991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:38075,127.17.110.61:35657,127.17.110.60:46033
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:17:46.447011 4826 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35657 } attempt: 1
I20260807 18:17:46.448386 4830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:46.447775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:35657 (local address 127.0.0.1:56436)
0807 18:17:46.447995 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:46.448170 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.61:35657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 18:17:46.449877 4830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:46.449459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:35657 (local address 127.0.0.1:56452)
0807 18:17:46.449611 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:46.449697 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.61:35657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 18:17:46.450083 4826 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.61:35657: Network error: Client connection negotiation failed: client connection to 127.17.110.61:35657: connect: Connection refused (error 111)
I20260807 18:17:46.518383 4826 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35657 } attempt: 2
I20260807 18:17:46.519804 4830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:46.519232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:35657 (local address 127.0.0.1:56458)
0807 18:17:46.519546 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:46.519643 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.61:35657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 18:17:46.520991 4830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:46.520577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:35657 (local address 127.0.0.1:56472)
0807 18:17:46.520753 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:46.520841 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.61:35657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 18:17:46.521195 4826 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.61:35657: Network error: Client connection negotiation failed: client connection to 127.17.110.61:35657: connect: Connection refused (error 111)
I20260807 18:17:46.612488 4826 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35657 } attempt: 3
I20260807 18:17:46.613719 4830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:46.613227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:35657 (local address 127.0.0.1:56476)
0807 18:17:46.613468 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:46.613583 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.61:35657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260807 18:17:46.614952 4830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:46.614497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:35657 (local address 127.0.0.1:56480)
0807 18:17:46.614664 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:46.614801 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.61:35657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 18:17:46.615142 4826 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.61:35657: Network error: Client connection negotiation failed: client connection to 127.17.110.61:35657: connect: Connection refused (error 111)
I20260807 18:17:46.758421 4826 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35657 } attempt: 4
I20260807 18:17:46.759886 4830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:46.759127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:35657 (local address 127.0.0.1:56496)
0807 18:17:46.759534 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:46.759673 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.61:35657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194}
I20260807 18:17:46.761154 4830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:46.760678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:35657 (local address 127.0.0.1:56498)
0807 18:17:46.760852 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:46.760972 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.61:35657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260807 18:17:46.761413 4826 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.61:35657: Network error: Client connection negotiation failed: client connection to 127.17.110.61:35657: connect: Connection refused (error 111)
W20260807 18:17:46.792833 4836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:46.793170 4836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:46.793352 4836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:46.800987 4836 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:17:46.801123 4836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:46.801256 4836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:46.801349 4836 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:17:46.801433 4836 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:17:46.812311 4836 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:37991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:38075,127.17.110.61:35657,127.17.110.60:46033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.61.keytab
--principal=kudu/127.17.110.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.61:35657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.110.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:17:46.813803 4836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:46.815307 4836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:46.827907 4844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:46.828018 4845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:46.828042 4847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:46.829254 4836 server_base.cc:1034] running on GCE node
I20260807 18:17:46.830113 4836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:46.831722 4836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:46.833068 4836 hybrid_clock.cc:648] HybridClock initialized: now 1786126666833007 us; error 48 us; skew 500 ppm
I20260807 18:17:46.833431 4836 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:46 dist-test-slave-39l8 krb5kdc[4714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126666, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:46.837069 4836 init.cc:377] Logged in from keytab as kudu/127.17.110.61@KRBTEST.COM (short username kudu)
I20260807 18:17:46.838816 4836 webserver.cc:460] Webserver started at http://127.17.110.61:40605/ using document root <none> and password file <none>
I20260807 18:17:46.839617 4836 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:46.839780 4836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:46.840119 4836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:46.843562 4836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-1/data/instance:
uuid: "1fd3a75a6c974391b574eb8a31ed4837"
format_stamp: "Formatted at 2026-08-07 18:17:46 on dist-test-slave-39l8"
server_key: "2f38c79872da7ddd0fb56b15f8ce7e8c"
server_key_iv: "5baee78df7790b536573bba988ca3a63"
server_key_version: "encryptionkey@0"
I20260807 18:17:46.844486 4836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-1/wal/instance:
uuid: "1fd3a75a6c974391b574eb8a31ed4837"
format_stamp: "Formatted at 2026-08-07 18:17:46 on dist-test-slave-39l8"
server_key: "2f38c79872da7ddd0fb56b15f8ce7e8c"
server_key_iv: "5baee78df7790b536573bba988ca3a63"
server_key_version: "encryptionkey@0"
I20260807 18:17:46.850425 4836 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.003s
I20260807 18:17:46.854542 4857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:46.856223 4836 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:17:46.856490 4836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-1/wal
uuid: "1fd3a75a6c974391b574eb8a31ed4837"
format_stamp: "Formatted at 2026-08-07 18:17:46 on dist-test-slave-39l8"
server_key: "2f38c79872da7ddd0fb56b15f8ce7e8c"
server_key_iv: "5baee78df7790b536573bba988ca3a63"
server_key_version: "encryptionkey@0"
I20260807 18:17:46.856729 4836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:46.872956 4836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:46.877969 4836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:46.878334 4836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:46.899466 4836 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.61:35657
I20260807 18:17:46.899503 4935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.61:35657 every 8 connection(s)
I20260807 18:17:46.901062 4836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-1/data/info.pb
I20260807 18:17:46.905174 4936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:46.908319 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 4836
I20260807 18:17:46.908529 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-1/wal/instance
I20260807 18:17:46.909013 17848 external_mini_cluster.cc:1428] Setting key 0512edb258f057f7259f413fd2e454a6
I20260807 18:17:46.912606 4936 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38075 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:17:46 dist-test-slave-39l8 krb5kdc[4714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126666, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.61@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.110.60@KRBTEST.COM; defaulting to no policy
I20260807 18:17:46.939131 4938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:46.915499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51330 (local address 127.17.110.62:38075)
0807 18:17:46.916129 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:46.916136 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:46.917214 (+ 1078us) server_negotiation.cc:407] Connection header received
0807 18:17:46.920297 (+ 3083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:46.920306 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:46.920419 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:46.920566 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:46.922296 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.923628 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.925386 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.926200 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.926375 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:46.930830 (+ 4455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:46.930841 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:46.930849 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:46.930910 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:46.935193 (+ 4283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:46.936222 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:46.936238 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:46.936242 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:46.936418 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:46.936902 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:46.936910 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:46.936915 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:46.937358 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:46.937551 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:46.938563 (+ 1012us) server_negotiation.cc:299] Negotiation successful
0807 18:17:46.938835 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":335,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.60@KRBTEST.COM" created.
I20260807 18:17:46.940536 4942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:46.916052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38075 (local address 127.0.0.1:51330)
0807 18:17:46.916995 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:46.917055 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:46.919898 (+ 2843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:46.920820 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:46.920829 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:46.921398 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:46.922048 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:46.922070 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.923875 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.923879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:46.925196 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:46.925212 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.926488 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.926494 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:46.926827 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:46.927639 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:17:46.927665 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:46.930451 (+ 2786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:46.935447 (+ 4996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:46.935457 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:46.935473 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:46.936050 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:46.936559 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:46.936563 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:46.936565 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:46.936764 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:46.937666 (+ 902us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:46.937678 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:46.938116 (+ 438us) client_negotiation.cc:769] Sending connection context
0807 18:17:46.938438 (+ 322us) client_negotiation.cc:240] Negotiation successful
0807 18:17:46.939054 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":339,"threads_started":1}
I20260807 18:17:46.942590 4936 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35657 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:17:46 dist-test-slave-39l8 krb5kdc[4714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126666, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.61@KRBTEST.COM for kudu/127.17.110.61@KRBTEST.COM
I20260807 18:17:46.959800 4944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:46.943865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56504 (local address 127.17.110.61:35657)
0807 18:17:46.944570 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:46.944578 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:46.944604 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:17:46.944653 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:46.944659 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:46.944864 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:46.945040 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:46.945873 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.947070 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.948230 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.948830 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.948973 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:46.952930 (+ 3957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:46.952946 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:46.952962 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:46.952997 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:46.956217 (+ 3220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:46.957250 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:46.957259 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:46.957262 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:46.957397 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:46.957912 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:46.957916 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:46.957918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:46.958383 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:46.958563 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:46.959127 (+ 564us) server_negotiation.cc:299] Negotiation successful
0807 18:17:46.959470 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":546,"thread_start_us":398,"threads_started":1}
I20260807 18:17:46.959860 4942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:46.943632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:35657 (local address 127.0.0.1:56504)
0807 18:17:46.943847 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:46.943868 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:46.943991 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:46.945028 (+ 1037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:46.945033 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:46.945545 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:46.945701 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:46.945710 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.947215 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.947223 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:46.948075 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:46.948089 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.949000 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.949006 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:46.949224 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:46.949950 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:17:46.949969 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:46.952645 (+ 2676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:46.956413 (+ 3768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:46.956422 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:46.956429 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:46.957063 (+ 634us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:46.957528 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:46.957536 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:46.957543 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:46.957765 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:46.958541 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:46.958549 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:46.958776 (+ 227us) client_negotiation.cc:769] Sending connection context
0807 18:17:46.958965 (+ 189us) client_negotiation.cc:240] Negotiation successful
0807 18:17:46.959341 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 18:17:46.963029 4936 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.110.60" port: 46033 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:17:46.964468 4942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:46.963811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:46033 (local address 127.0.0.1:44252)
0807 18:17:46.964039 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:46.964216 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260807 18:17:46.964845 4867 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.110.60:46033: connect: Connection refused (error 111)
I20260807 18:17:46.966931 4942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:46.966379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:46033 (local address 127.0.0.1:44258)
0807 18:17:46.966580 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:46.966681 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260807 18:17:46.967280 4936 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.60:46033: Network error: Client connection negotiation failed: client connection to 127.17.110.60:46033: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.60.keytab.
Entry for principal kudu/127.17.110.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.60.keytab.
I20260807 18:17:46.980651 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.60:46033
--webserver_interface=127.17.110.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.60.keytab
--principal=kudu/127.17.110.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:37991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:38075,127.17.110.61:35657,127.17.110.60:46033
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:17:47.016738 4936 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.60" port: 46033 } attempt: 1
I20260807 18:17:47.018062 4942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.017541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:46033 (local address 127.0.0.1:44262)
0807 18:17:47.017770 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:47.017890 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 18:17:47.019508 4942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.019023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:46033 (local address 127.0.0.1:44278)
0807 18:17:47.019248 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:47.019350 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260807 18:17:47.019781 4936 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.60:46033: Network error: Client connection negotiation failed: client connection to 127.17.110.60:46033: connect: Connection refused (error 111)
I20260807 18:17:47.060823 4826 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35657 } attempt: 5
Aug 07 18:17:47 dist-test-slave-39l8 krb5kdc[4714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126666, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for kudu/127.17.110.61@KRBTEST.COM
I20260807 18:17:47.079718 4830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.061885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:35657 (local address 127.0.0.1:56510)
0807 18:17:47.062117 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:47.062144 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:47.062348 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:47.062899 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:47.062904 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:47.063524 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:47.063742 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:47.063753 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.065294 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.065300 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:47.066311 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:47.066329 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.067557 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.067564 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:47.067744 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:47.068584 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:17:47.068607 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:47.071296 (+ 2689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:47.076442 (+ 5146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:47.076448 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:47.076452 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:47.076925 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:47.077609 (+ 684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:47.077614 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:47.077616 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:47.077829 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:47.078764 (+ 935us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:47.078774 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:47.078962 (+ 188us) client_negotiation.cc:769] Sending connection context
0807 18:17:47.079134 (+ 172us) client_negotiation.cc:240] Negotiation successful
0807 18:17:47.079409 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 18:17:47.080354 4944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.061885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56510 (local address 127.17.110.61:35657)
0807 18:17:47.062200 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:47.062209 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:47.062279 (+ 70us) server_negotiation.cc:407] Connection header received
0807 18:17:47.062504 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:47.062510 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:47.062603 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:47.062784 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:47.063968 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.065075 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.066535 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.067317 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.067496 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:47.071726 (+ 4230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:47.071735 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:47.071740 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:47.071819 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:47.076098 (+ 4279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:47.077127 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:47.077137 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:47.077143 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:47.077376 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:47.078010 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:47.078019 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:47.078024 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:47.078491 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:47.078658 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:47.079316 (+ 658us) server_negotiation.cc:299] Negotiation successful
0807 18:17:47.079642 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260807 18:17:47.081861 4826 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.110.60" port: 46033 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:17:47.083391 4830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.082920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:46033 (local address 127.0.0.1:44290)
0807 18:17:47.083119 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:47.083226 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 18:17:47.085346 4830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.084504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:46033 (local address 127.0.0.1:44294)
0807 18:17:47.084887 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:47.085047 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239}
W20260807 18:17:47.085681 4826 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.60:46033: Network error: Client connection negotiation failed: client connection to 127.17.110.60:46033: connect: Connection refused (error 111)
I20260807 18:17:47.088047 4936 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.60" port: 46033 } attempt: 2
I20260807 18:17:47.089408 4942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.088847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:46033 (local address 127.0.0.1:44308)
0807 18:17:47.089106 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:47.089213 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260807 18:17:47.090878 4942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.090305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:46033 (local address 127.0.0.1:44320)
0807 18:17:47.090497 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:47.090630 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260807 18:17:47.091166 4936 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.60:46033: Network error: Client connection negotiation failed: client connection to 127.17.110.60:46033: connect: Connection refused (error 111)
I20260807 18:17:47.137006 4826 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.60" port: 46033 } attempt: 1
I20260807 18:17:47.138566 4830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.137866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:46033 (local address 127.0.0.1:44332)
0807 18:17:47.138163 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:47.138319 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260807 18:17:47.139863 4830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.139418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:46033 (local address 127.0.0.1:44338)
0807 18:17:47.139608 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:47.139707 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260807 18:17:47.140144 4826 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.60:46033: Network error: Client connection negotiation failed: client connection to 127.17.110.60:46033: connect: Connection refused (error 111)
I20260807 18:17:47.182451 4936 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.60" port: 46033 } attempt: 3
I20260807 18:17:47.183739 4942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.183216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:46033 (local address 127.0.0.1:44354)
0807 18:17:47.183454 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:47.183564 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260807 18:17:47.185142 4942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.184594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:46033 (local address 127.0.0.1:44362)
0807 18:17:47.184867 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:47.184954 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
W20260807 18:17:47.185395 4936 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.60:46033: Network error: Client connection negotiation failed: client connection to 127.17.110.60:46033: connect: Connection refused (error 111)
I20260807 18:17:47.208388 4826 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.60" port: 46033 } attempt: 2
I20260807 18:17:47.209725 4830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.209125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:46033 (local address 127.0.0.1:44368)
0807 18:17:47.209388 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:47.209507 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260807 18:17:47.210947 4830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.210484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:46033 (local address 127.0.0.1:44382)
0807 18:17:47.210657 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:47.210781 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 18:17:47.211227 4826 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.60:46033: Network error: Client connection negotiation failed: client connection to 127.17.110.60:46033: connect: Connection refused (error 111)
I20260807 18:17:47.317498 4826 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.60" port: 46033 } attempt: 3
I20260807 18:17:47.319075 4830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.318359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:46033 (local address 127.0.0.1:44394)
0807 18:17:47.318671 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:47.318836 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 18:17:47.320576 4830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.320020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:46033 (local address 127.0.0.1:44410)
0807 18:17:47.320293 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:47.320403 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
W20260807 18:17:47.320909 4826 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.60:46033: Network error: Client connection negotiation failed: client connection to 127.17.110.60:46033: connect: Connection refused (error 111)
I20260807 18:17:47.328707 4936 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.60" port: 46033 } attempt: 4
I20260807 18:17:47.330099 4942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.329511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:46033 (local address 127.0.0.1:44414)
0807 18:17:47.329803 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:47.329900 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260807 18:17:47.331400 4942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.330989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:46033 (local address 127.0.0.1:44426)
0807 18:17:47.331167 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:47.331271 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:46033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 18:17:47.331662 4936 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.60:46033: Network error: Client connection negotiation failed: client connection to 127.17.110.60:46033: connect: Connection refused (error 111)
W20260807 18:17:47.372067 4947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:47.372480 4947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:47.372651 4947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:47.381261 4947 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:17:47.381446 4947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:47.381567 4947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:47.381682 4947 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:17:47.381767 4947 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:17:47.392653 4947 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:37991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:38075,127.17.110.61:35657,127.17.110.60:46033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.60.keytab
--principal=kudu/127.17.110.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.60:46033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.110.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:17:47.394205 4947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:47.395661 4947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:47.408486 4955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:47.409088 4959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:47.409126 4956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:47.410070 4947 server_base.cc:1034] running on GCE node
I20260807 18:17:47.411089 4947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:47.412719 4947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:47.413990 4947 hybrid_clock.cc:648] HybridClock initialized: now 1786126667413931 us; error 56 us; skew 500 ppm
I20260807 18:17:47.414445 4947 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:47 dist-test-slave-39l8 krb5kdc[4714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126667, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:47.417954 4947 init.cc:377] Logged in from keytab as kudu/127.17.110.60@KRBTEST.COM (short username kudu)
I20260807 18:17:47.419811 4947 webserver.cc:460] Webserver started at http://127.17.110.60:40655/ using document root <none> and password file <none>
I20260807 18:17:47.420661 4947 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:47.420828 4947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:47.421185 4947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:47.424573 4947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-2/data/instance:
uuid: "2d9b46ff786c4955bc9afc8c51722f2e"
format_stamp: "Formatted at 2026-08-07 18:17:47 on dist-test-slave-39l8"
server_key: "873c911523873b9991ad7d20707f4ded"
server_key_iv: "a588d05e999b5c7cfb3ca1c3ee143878"
server_key_version: "encryptionkey@0"
I20260807 18:17:47.425510 4947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-2/wal/instance:
uuid: "2d9b46ff786c4955bc9afc8c51722f2e"
format_stamp: "Formatted at 2026-08-07 18:17:47 on dist-test-slave-39l8"
server_key: "873c911523873b9991ad7d20707f4ded"
server_key_iv: "a588d05e999b5c7cfb3ca1c3ee143878"
server_key_version: "encryptionkey@0"
I20260807 18:17:47.431720 4947 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260807 18:17:47.436194 4968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:47.437942 4947 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260807 18:17:47.438210 4947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-2/wal
uuid: "2d9b46ff786c4955bc9afc8c51722f2e"
format_stamp: "Formatted at 2026-08-07 18:17:47 on dist-test-slave-39l8"
server_key: "873c911523873b9991ad7d20707f4ded"
server_key_iv: "a588d05e999b5c7cfb3ca1c3ee143878"
server_key_version: "encryptionkey@0"
I20260807 18:17:47.438472 4947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:47.455260 4947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:47.460934 4947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:47.461333 4947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:47.483969 4947 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.60:46033
I20260807 18:17:47.483991 5042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.60:46033 every 8 connection(s)
I20260807 18:17:47.485697 4947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-2/data/info.pb
I20260807 18:17:47.490010 5044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:47.493088 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 4947
I20260807 18:17:47.493310 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126429568977-17848-0/minicluster-data/master-2/wal/instance
I20260807 18:17:47.493876 17848 external_mini_cluster.cc:1428] Setting key ad16bb3f09ad11b3bb87570a5a5567c7
I20260807 18:17:47.498207 4826 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.60" port: 46033 } attempt: 4
I20260807 18:17:47.499495 5044 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38075 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:17:47 dist-test-slave-39l8 krb5kdc[4714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126666, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for kudu/127.17.110.60@KRBTEST.COM
Aug 07 18:17:47 dist-test-slave-39l8 krb5kdc[4714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.60@KRBTEST.COM
Aug 07 18:17:47 dist-test-slave-39l8 krb5kdc[4714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126667, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.60@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:17:47.525103 4830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.499000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:46033 (local address 127.0.0.1:44444)
0807 18:17:47.499333 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:47.499356 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:47.499489 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:47.501084 (+ 1595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:47.501090 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:47.501665 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:47.501879 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:47.501893 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.504772 (+ 2879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.504779 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:47.506277 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:47.506300 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.508990 (+ 2690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.508996 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:47.509247 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:47.509899 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:17:47.509927 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:47.512999 (+ 3072us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:47.521117 (+ 8118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:47.521125 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:47.521128 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:47.521824 (+ 696us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:47.522532 (+ 708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:47.522537 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:47.522541 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:47.522750 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:47.523947 (+ 1197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:47.523953 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:47.524170 (+ 217us) client_negotiation.cc:769] Sending connection context
0807 18:17:47.524380 (+ 210us) client_negotiation.cc:240] Negotiation successful
0807 18:17:47.524728 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197}
I20260807 18:17:47.526877 5051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.499178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44444 (local address 127.17.110.60:46033)
0807 18:17:47.500010 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:47.500014 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:47.500068 (+ 54us) server_negotiation.cc:407] Connection header received
0807 18:17:47.500125 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:47.500130 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:47.500648 (+ 518us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:47.501678 (+ 1030us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:47.502119 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.504522 (+ 2403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.506526 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.508762 (+ 2236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.508921 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:47.513325 (+ 4404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:47.513339 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:47.513361 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:47.513420 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:47.518982 (+ 5562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:47.522050 (+ 3068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:47.522061 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:47.522072 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:47.522361 (+ 289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:47.522915 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:47.522920 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:47.522923 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:47.523722 (+ 799us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:47.523956 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:47.524664 (+ 708us) server_negotiation.cc:299] Negotiation successful
0807 18:17:47.525205 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1367,"server-negotiator.queue_time_us":711,"spinlock_wait_cycles":20352,"thread_start_us":556,"threads_started":1}
I20260807 18:17:47.526985 5050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.496404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44434 (local address 127.17.110.60:46033)
0807 18:17:47.497346 (+ 942us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:47.497360 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:47.497407 (+ 47us) server_negotiation.cc:407] Connection header received
0807 18:17:47.500094 (+ 2687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:47.500121 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:47.500645 (+ 524us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:47.501085 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:47.503315 (+ 2230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.504524 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.506635 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.507847 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.508061 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:47.516224 (+ 8163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:47.516232 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:47.516237 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:47.516285 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:47.523043 (+ 6758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:47.524183 (+ 1140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:47.524188 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:47.524251 (+ 63us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:47.524398 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:47.524933 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:47.524939 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:47.524943 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:47.525361 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:47.525627 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:47.526277 (+ 650us) server_negotiation.cc:299] Negotiation successful
0807 18:17:47.526516 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":696,"spinlock_wait_cycles":10496,"thread_start_us":423,"threads_started":1}
I20260807 18:17:47.533973 5054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.503621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:38075 (local address 127.0.0.1:51336)
0807 18:17:47.507546 (+ 3925us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:47.507596 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:47.507795 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:47.508737 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:47.508747 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:47.509739 (+ 992us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:47.510012 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:47.510021 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.511225 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.511229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:47.512823 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:47.512841 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.513721 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.513731 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:47.513963 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:47.521845 (+ 7882us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:17:47.521884 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:47.525504 (+ 3620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:47.530618 (+ 5114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:47.530631 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:47.530639 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:47.531397 (+ 758us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:47.531860 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:47.531865 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:47.531869 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:47.532039 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:47.532708 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:47.532716 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:47.532906 (+ 190us) client_negotiation.cc:769] Sending connection context
0807 18:17:47.533104 (+ 198us) client_negotiation.cc:240] Negotiation successful
0807 18:17:47.533482 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3637,"thread_start_us":3455,"threads_started":1}
I20260807 18:17:47.533982 5052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.503452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51336 (local address 127.17.110.62:38075)
0807 18:17:47.508339 (+ 4887us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:47.508347 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:47.508366 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:17:47.508436 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:47.508439 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:47.508503 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:47.508639 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:47.510180 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.511009 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.512999 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.513510 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.513659 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:47.525784 (+ 12125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:47.525790 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:47.525794 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:47.525836 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:47.530423 (+ 4587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:47.531572 (+ 1149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:47.531576 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:47.531579 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:47.531722 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:47.532184 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:47.532189 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:47.532192 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:47.532560 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:47.532743 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:47.533242 (+ 499us) server_negotiation.cc:299] Negotiation successful
0807 18:17:47.533482 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4716,"thread_start_us":4581,"threads_started":1}
I20260807 18:17:47.535421 4826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c: Bootstrap starting.
I20260807 18:17:47.536072 5044 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35657 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:17:47.540205 4826 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:47.541802 4826 log.cc:824] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:17:47 dist-test-slave-39l8 krb5kdc[4714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
Aug 07 18:17:47 dist-test-slave-39l8 krb5kdc[4714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126667, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.60@KRBTEST.COM for kudu/127.17.110.61@KRBTEST.COM
I20260807 18:17:47.546377 4826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c: No bootstrap required, opened a new log
I20260807 18:17:47.551749 5054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.536800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:35657 (local address 127.0.0.1:56526)
0807 18:17:47.537340 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:47.537365 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:47.537489 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:47.537999 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:47.538002 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:47.538383 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:47.538523 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:47.538531 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.539602 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.539612 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:47.540590 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:47.540604 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.541355 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.541361 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:47.541560 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:47.542266 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:17:47.542294 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:47.544911 (+ 2617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:47.548486 (+ 3575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:47.548491 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:47.548494 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:47.548971 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:47.549508 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:47.549512 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:47.549514 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:47.549759 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:47.550510 (+ 751us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:47.550517 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:47.550753 (+ 236us) client_negotiation.cc:769] Sending connection context
0807 18:17:47.550967 (+ 214us) client_negotiation.cc:240] Negotiation successful
0807 18:17:47.551317 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392}
I20260807 18:17:47.551776 4944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.537260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56526 (local address 127.17.110.61:35657)
0807 18:17:47.537456 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:47.537462 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:47.537483 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:17:47.537629 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:47.537637 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:47.537723 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:47.537824 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:47.538698 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.539430 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.540753 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.541183 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.541337 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:47.545164 (+ 3827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:47.545170 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:47.545173 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:47.545213 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:47.548275 (+ 3062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:47.549129 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:47.549139 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:47.549146 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:47.549346 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:47.549919 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:47.549925 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:47.549928 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:47.550342 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:47.550535 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:47.551100 (+ 565us) server_negotiation.cc:299] Negotiation successful
0807 18:17:47.551355 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 18:17:47.553419 5044 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.110.60" port: 46033 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:17:47.556504 5055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.530932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51352 (local address 127.17.110.62:38075)
0807 18:17:47.531624 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:47.531629 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:47.531657 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:17:47.531722 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:47.531734 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:47.531817 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:47.533709 (+ 1892us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:47.534077 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.535416 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.536852 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.537492 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.537700 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:47.543834 (+ 6134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:47.543841 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:47.543846 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:47.543891 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:47.552730 (+ 8839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:47.553754 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:47.553760 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:47.553763 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:47.554010 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:47.554487 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:47.554493 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:47.554496 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:47.555009 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:47.555342 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:47.555715 (+ 373us) server_negotiation.cc:299] Negotiation successful
0807 18:17:47.555989 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":299,"threads_started":1}
I20260807 18:17:47.558055 4826 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4a971bf593423592de0955a984311c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38075 } } peers { permanent_uuid: "1fd3a75a6c974391b574eb8a31ed4837" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35657 } } peers { permanent_uuid: "2d9b46ff786c4955bc9afc8c51722f2e" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 46033 } }
I20260807 18:17:47.558665 4826 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:47.558878 4826 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f4a971bf593423592de0955a984311c, State: Initialized, Role: FOLLOWER
I20260807 18:17:47.559824 4826 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4a971bf593423592de0955a984311c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38075 } } peers { permanent_uuid: "1fd3a75a6c974391b574eb8a31ed4837" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35657 } } peers { permanent_uuid: "2d9b46ff786c4955bc9afc8c51722f2e" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 46033 } }
I20260807 18:17:47.561103 5057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4a971bf593423592de0955a984311c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38075 } } peers { permanent_uuid: "1fd3a75a6c974391b574eb8a31ed4837" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35657 } } peers { permanent_uuid: "2d9b46ff786c4955bc9afc8c51722f2e" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 46033 } } }
I20260807 18:17:47.561893 5057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:17:47.562738 4826 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c [sys.catalog]: configured and running, proceeding with master startup.
Aug 07 18:17:47 dist-test-slave-39l8 krb5kdc[4714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126667, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.60@KRBTEST.COM for kudu/127.17.110.60@KRBTEST.COM
Aug 07 18:17:47 dist-test-slave-39l8 krb5kdc[4714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.61@KRBTEST.COM
I20260807 18:17:47.574512 5054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.554371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:46033 (local address 127.0.0.1:44458)
0807 18:17:47.554756 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:47.554778 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:47.554887 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:47.555392 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:47.555399 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:47.555929 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:47.556335 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:47.556355 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.557767 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.557772 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:47.559142 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:47.559160 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.560379 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.560387 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:47.560656 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:47.561433 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:17:47.561458 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:47.564770 (+ 3312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:47.570802 (+ 6032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:47.570809 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:47.570814 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:47.571524 (+ 710us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:47.572055 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:47.572073 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:47.572080 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:47.572281 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:47.573223 (+ 942us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:47.573228 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:47.573428 (+ 200us) client_negotiation.cc:769] Sending connection context
0807 18:17:47.573629 (+ 201us) client_negotiation.cc:240] Negotiation successful
0807 18:17:47.574033 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225}
I20260807 18:17:47.575718 5050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.554745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44458 (local address 127.17.110.60:46033)
0807 18:17:47.554888 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:47.554894 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:47.554911 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:17:47.555032 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:47.555038 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:47.555126 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:47.555217 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:47.556518 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.557586 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.559312 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.559926 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.560034 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:47.565039 (+ 5005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:47.565050 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:47.565058 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:47.565115 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:47.570489 (+ 5374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:47.571696 (+ 1207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:47.571701 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:47.571707 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:47.571885 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:47.572451 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:47.572474 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:47.572480 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:47.572978 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:47.573168 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:47.573736 (+ 568us) server_negotiation.cc:299] Negotiation successful
0807 18:17:47.575267 (+ 1531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1257,"server-negotiator.queue_time_us":43}
I20260807 18:17:47.580264 4944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.558465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56536 (local address 127.17.110.61:35657)
0807 18:17:47.558651 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:47.558657 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:47.558760 (+ 103us) server_negotiation.cc:407] Connection header received
0807 18:17:47.558969 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:47.558976 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:47.559073 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:47.559194 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:47.561024 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.562285 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.564666 (+ 2381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.565314 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.565402 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:47.571317 (+ 5915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:47.571328 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:47.571336 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:47.571396 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:47.575765 (+ 4369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:47.577113 (+ 1348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:47.577120 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:47.577123 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:47.577341 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:47.577987 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:47.577995 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:47.578000 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:47.578443 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:47.578817 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:47.579571 (+ 754us) server_negotiation.cc:299] Negotiation successful
0807 18:17:47.579862 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:17:47.582195 5044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d9b46ff786c4955bc9afc8c51722f2e: Bootstrap starting.
W20260807 18:17:47.582741 4759 tablet.cc:2369] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:17:47.587054 5044 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d9b46ff786c4955bc9afc8c51722f2e: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:47.587879 17848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:17:47.589053 5044 log.cc:824] T 00000000000000000000000000000000 P 2d9b46ff786c4955bc9afc8c51722f2e: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:47.593271 5044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d9b46ff786c4955bc9afc8c51722f2e: No bootstrap required, opened a new log
W20260807 18:17:47.593624 5073 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:17:47.593848 5073 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:17:47.606564 5044 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d9b46ff786c4955bc9afc8c51722f2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4a971bf593423592de0955a984311c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38075 } } peers { permanent_uuid: "1fd3a75a6c974391b574eb8a31ed4837" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35657 } } peers { permanent_uuid: "2d9b46ff786c4955bc9afc8c51722f2e" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 46033 } }
I20260807 18:17:47.607380 5044 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d9b46ff786c4955bc9afc8c51722f2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:47.607604 5044 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d9b46ff786c4955bc9afc8c51722f2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d9b46ff786c4955bc9afc8c51722f2e, State: Initialized, Role: FOLLOWER
I20260807 18:17:47.608649 5044 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d9b46ff786c4955bc9afc8c51722f2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4a971bf593423592de0955a984311c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38075 } } peers { permanent_uuid: "1fd3a75a6c974391b574eb8a31ed4837" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35657 } } peers { permanent_uuid: "2d9b46ff786c4955bc9afc8c51722f2e" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 46033 } }
I20260807 18:17:47.615063 5081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d9b46ff786c4955bc9afc8c51722f2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4a971bf593423592de0955a984311c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38075 } } peers { permanent_uuid: "1fd3a75a6c974391b574eb8a31ed4837" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35657 } } peers { permanent_uuid: "2d9b46ff786c4955bc9afc8c51722f2e" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 46033 } } }
I20260807 18:17:47.615710 5081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d9b46ff786c4955bc9afc8c51722f2e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:17:47.616856 5044 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d9b46ff786c4955bc9afc8c51722f2e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:17:47.598884 4732 master_runner.cc:186] Error getting master registration for 127.17.110.61:35657: OK, instance_id { permanent_uuid: "1fd3a75a6c974391b574eb8a31ed4837" instance_seqno: 1786126666893663 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:17:47.621094 4732 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 18:17:47.630865 4936 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.60" port: 46033 } attempt: 5
Aug 07 18:17:47 dist-test-slave-39l8 krb5kdc[4714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126666, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.61@KRBTEST.COM for kudu/127.17.110.60@KRBTEST.COM
I20260807 18:17:47.653949 4942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.631627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:46033 (local address 127.0.0.1:44472)
0807 18:17:47.631977 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:47.631998 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:47.632146 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:47.633201 (+ 1055us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:47.633207 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:47.634005 (+ 798us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:47.634287 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:47.634304 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.636193 (+ 1889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.636201 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:47.637906 (+ 1705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:47.637928 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.639583 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.639590 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:47.639901 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:47.640751 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:17:47.640781 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:47.644574 (+ 3793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:47.649755 (+ 5181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:47.649762 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:47.649767 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:47.650459 (+ 692us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:47.651064 (+ 605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:47.651070 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:47.651074 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:47.651272 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:47.652196 (+ 924us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:47.652202 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:47.652873 (+ 671us) client_negotiation.cc:769] Sending connection context
0807 18:17:47.653080 (+ 207us) client_negotiation.cc:240] Negotiation successful
0807 18:17:47.653453 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232}
I20260807 18:17:47.655690 5051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.632488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44472 (local address 127.17.110.60:46033)
0807 18:17:47.632690 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:47.632696 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:47.632728 (+ 32us) server_negotiation.cc:407] Connection header received
0807 18:17:47.632795 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:47.632800 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:47.632886 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:47.632991 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:47.634648 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.635962 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.638146 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.639172 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.639327 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:47.644916 (+ 5589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:47.644926 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:47.644932 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:47.644985 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:47.649426 (+ 4441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:47.650678 (+ 1252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:47.650685 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:47.650690 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:47.650894 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:47.651428 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:47.651433 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:47.651437 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:47.651898 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:47.652098 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:47.654989 (+ 2891us) server_negotiation.cc:299] Negotiation successful
0807 18:17:47.655289 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:17:47.663264 4936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fd3a75a6c974391b574eb8a31ed4837: Bootstrap starting.
I20260807 18:17:47.668438 4936 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1fd3a75a6c974391b574eb8a31ed4837: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:47.670528 4936 log.cc:824] T 00000000000000000000000000000000 P 1fd3a75a6c974391b574eb8a31ed4837: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:47.684441 4936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fd3a75a6c974391b574eb8a31ed4837: No bootstrap required, opened a new log
I20260807 18:17:47.703277 4936 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1fd3a75a6c974391b574eb8a31ed4837 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4a971bf593423592de0955a984311c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38075 } } peers { permanent_uuid: "1fd3a75a6c974391b574eb8a31ed4837" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35657 } } peers { permanent_uuid: "2d9b46ff786c4955bc9afc8c51722f2e" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 46033 } }
I20260807 18:17:47.704031 4936 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1fd3a75a6c974391b574eb8a31ed4837 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:47.704345 4936 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1fd3a75a6c974391b574eb8a31ed4837 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fd3a75a6c974391b574eb8a31ed4837, State: Initialized, Role: FOLLOWER
I20260807 18:17:47.705438 4936 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fd3a75a6c974391b574eb8a31ed4837 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4a971bf593423592de0955a984311c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38075 } } peers { permanent_uuid: "1fd3a75a6c974391b574eb8a31ed4837" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35657 } } peers { permanent_uuid: "2d9b46ff786c4955bc9afc8c51722f2e" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 46033 } }
I20260807 18:17:47.706600 5057 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:17:47.706979 5057 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4a971bf593423592de0955a984311c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38075 } } peers { permanent_uuid: "1fd3a75a6c974391b574eb8a31ed4837" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35657 } } peers { permanent_uuid: "2d9b46ff786c4955bc9afc8c51722f2e" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 46033 } }
I20260807 18:17:47.708910 5057 leader_election.cc:288] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fd3a75a6c974391b574eb8a31ed4837 (127.17.110.61:35657), 2d9b46ff786c4955bc9afc8c51722f2e (127.17.110.60:46033)
I20260807 18:17:47.709867 5055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.622381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51356 (local address 127.17.110.62:38075)
0807 18:17:47.622628 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:47.622635 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:47.671928 (+ 49293us) server_negotiation.cc:407] Connection header received
0807 18:17:47.673465 (+ 1537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:47.673473 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:47.673577 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:47.673679 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:47.690810 (+ 17131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.692093 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.694496 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.695351 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.695523 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:47.699889 (+ 4366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:47.699898 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:47.699904 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:47.699969 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:47.704667 (+ 4698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:47.705977 (+ 1310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:47.705984 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:47.705989 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:47.706175 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:47.706956 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:47.706970 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:47.706976 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:47.707429 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:47.709017 (+ 1588us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:47.709181 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 18:17:47.709472 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 18:17:47.710412 5005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5f4a971bf593423592de0955a984311c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d9b46ff786c4955bc9afc8c51722f2e" is_pre_election: true
I20260807 18:17:47.711189 5005 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2d9b46ff786c4955bc9afc8c51722f2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f4a971bf593423592de0955a984311c in term 0.
I20260807 18:17:47.713217 4756 leader_election.cc:302] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d9b46ff786c4955bc9afc8c51722f2e, 5f4a971bf593423592de0955a984311c; no voters:
I20260807 18:17:47.713871 5057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:17:47.714043 5057 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:17:47.714164 5057 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c [term 0 FOLLOWER]: Advancing to term 1
W20260807 18:17:47.714179 4975 tablet.cc:2369] T 00000000000000000000000000000000 P 2d9b46ff786c4955bc9afc8c51722f2e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:17:47.717895 5057 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4a971bf593423592de0955a984311c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38075 } } peers { permanent_uuid: "1fd3a75a6c974391b574eb8a31ed4837" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35657 } } peers { permanent_uuid: "2d9b46ff786c4955bc9afc8c51722f2e" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 46033 } }
I20260807 18:17:47.720502 5005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5f4a971bf593423592de0955a984311c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d9b46ff786c4955bc9afc8c51722f2e"
I20260807 18:17:47.720789 5005 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d9b46ff786c4955bc9afc8c51722f2e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:47.727196 5005 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2d9b46ff786c4955bc9afc8c51722f2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f4a971bf593423592de0955a984311c in term 1.
I20260807 18:17:47.728333 4756 leader_election.cc:302] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d9b46ff786c4955bc9afc8c51722f2e, 5f4a971bf593423592de0955a984311c; no voters:
I20260807 18:17:47.732976 5057 leader_election.cc:288] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c [CANDIDATE]: Term 1 election: Requested vote from peers 1fd3a75a6c974391b574eb8a31ed4837 (127.17.110.61:35657), 2d9b46ff786c4955bc9afc8c51722f2e (127.17.110.60:46033)
I20260807 18:17:47.734459 5057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:17:47.735451 5057 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c [term 1 LEADER]: Becoming Leader. State: Replica: 5f4a971bf593423592de0955a984311c, State: Running, Role: LEADER
I20260807 18:17:47.740065 5057 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4a971bf593423592de0955a984311c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38075 } } peers { permanent_uuid: "1fd3a75a6c974391b574eb8a31ed4837" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35657 } } peers { permanent_uuid: "2d9b46ff786c4955bc9afc8c51722f2e" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 46033 } }
I20260807 18:17:47.754658 5106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fd3a75a6c974391b574eb8a31ed4837 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4a971bf593423592de0955a984311c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38075 } } peers { permanent_uuid: "1fd3a75a6c974391b574eb8a31ed4837" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35657 } } peers { permanent_uuid: "2d9b46ff786c4955bc9afc8c51722f2e" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 46033 } } }
I20260807 18:17:47.755295 5106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fd3a75a6c974391b574eb8a31ed4837 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:17:47.756007 4905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5f4a971bf593423592de0955a984311c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fd3a75a6c974391b574eb8a31ed4837" is_pre_election: true
I20260807 18:17:47.756891 4903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5f4a971bf593423592de0955a984311c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fd3a75a6c974391b574eb8a31ed4837"
I20260807 18:17:47.757584 4944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.629860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56538 (local address 127.17.110.61:35657)
0807 18:17:47.630069 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:47.630076 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:47.679370 (+ 49294us) server_negotiation.cc:407] Connection header received
0807 18:17:47.679587 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:47.679595 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:47.679714 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:47.679835 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:47.685703 (+ 5868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.687058 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.714407 (+ 27349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.715294 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.715424 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:47.729879 (+ 14455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:47.729892 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:47.729902 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:47.729977 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:47.735417 (+ 5440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:47.745575 (+ 10158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:47.745583 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:47.745587 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:47.746038 (+ 451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:47.748441 (+ 2403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:47.748472 (+ 31us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:47.748475 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:47.749367 (+ 892us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:47.749921 (+ 554us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:47.756285 (+ 6364us) server_negotiation.cc:299] Negotiation successful
0807 18:17:47.756761 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 18:17:47.757619 4936 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fd3a75a6c974391b574eb8a31ed4837 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:17:47.756798 4905 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1fd3a75a6c974391b574eb8a31ed4837 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f4a971bf593423592de0955a984311c in term 0.
I20260807 18:17:47.760274 5050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.624103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44466 (local address 127.17.110.60:46033)
0807 18:17:47.626952 (+ 2849us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:47.626959 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:47.678650 (+ 51691us) server_negotiation.cc:407] Connection header received
0807 18:17:47.680640 (+ 1990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:47.680647 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:47.680744 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:47.680863 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:47.689377 (+ 8514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.690689 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.716711 (+ 26022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.717538 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.717683 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:47.737111 (+ 19428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:47.737122 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:47.737128 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:47.737191 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:47.742026 (+ 4835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:47.747104 (+ 5078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:47.747113 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:47.747119 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:47.747367 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:47.750607 (+ 3240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:47.750615 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:47.750618 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:47.751593 (+ 975us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:47.752184 (+ 591us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:47.759415 (+ 7231us) server_negotiation.cc:299] Negotiation successful
0807 18:17:47.759757 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2658}
I20260807 18:17:47.766304 5108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f4a971bf593423592de0955a984311c. Latest consensus state: current_term: 1 leader_uuid: "5f4a971bf593423592de0955a984311c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4a971bf593423592de0955a984311c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38075 } } peers { permanent_uuid: "1fd3a75a6c974391b574eb8a31ed4837" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35657 } } peers { permanent_uuid: "2d9b46ff786c4955bc9afc8c51722f2e" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 46033 } } }
I20260807 18:17:47.769410 5108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c [sys.catalog]: This master's current role is: LEADER
I20260807 18:17:47.770623 5117 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:17:47.775864 5117 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:17:47.795075 5005 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2d9b46ff786c4955bc9afc8c51722f2e [term 1 FOLLOWER]: Refusing update from remote peer 5f4a971bf593423592de0955a984311c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:17:47.796391 5057 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d9b46ff786c4955bc9afc8c51722f2e" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 46033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:17:47.802186 4905 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1fd3a75a6c974391b574eb8a31ed4837 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:47.806771 4905 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1fd3a75a6c974391b574eb8a31ed4837 [term 1 FOLLOWER]: Refusing update from remote peer 5f4a971bf593423592de0955a984311c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:17:47.807992 5057 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fd3a75a6c974391b574eb8a31ed4837" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:17:47.786286 4947 master_runner.cc:186] Error getting master registration for 127.17.110.61:35657: OK, instance_id { permanent_uuid: "1fd3a75a6c974391b574eb8a31ed4837" instance_seqno: 1786126666893663 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:17:47.819600 4947 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 18:17:47.832485 5081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d9b46ff786c4955bc9afc8c51722f2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f4a971bf593423592de0955a984311c. Latest consensus state: current_term: 1 leader_uuid: "5f4a971bf593423592de0955a984311c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4a971bf593423592de0955a984311c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38075 } } peers { permanent_uuid: "1fd3a75a6c974391b574eb8a31ed4837" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35657 } } peers { permanent_uuid: "2d9b46ff786c4955bc9afc8c51722f2e" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 46033 } } }
I20260807 18:17:47.833007 5081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d9b46ff786c4955bc9afc8c51722f2e [sys.catalog]: This master's current role is: FOLLOWER
W20260807 18:17:47.851965 5124 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2d9b46ff786c4955bc9afc8c51722f2e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:17:47.852216 5124 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:17:47.863284 5129 mvcc.cc:205] Tried to move back new op lower bound from 7315974831273664512 to 7315974831109234688. Current Snapshot: MvccSnapshot[applied={T|T < 7315974831273664512}]
I20260807 18:17:47.868616 5057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5f4a971bf593423592de0955a984311c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4a971bf593423592de0955a984311c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38075 } } peers { permanent_uuid: "1fd3a75a6c974391b574eb8a31ed4837" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35657 } } peers { permanent_uuid: "2d9b46ff786c4955bc9afc8c51722f2e" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 46033 } } }
I20260807 18:17:47.869065 5057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c [sys.catalog]: This master's current role is: LEADER
I20260807 18:17:47.875140 5117 catalog_manager.cc:1347] Generated new cluster ID: 8dbf3a0830c34e02a2d177c19455604e
I20260807 18:17:47.875296 5117 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
W20260807 18:17:47.881731 4873 tablet.cc:2369] T 00000000000000000000000000000000 P 1fd3a75a6c974391b574eb8a31ed4837: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:17:47.891924 5081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d9b46ff786c4955bc9afc8c51722f2e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5f4a971bf593423592de0955a984311c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4a971bf593423592de0955a984311c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38075 } } peers { permanent_uuid: "1fd3a75a6c974391b574eb8a31ed4837" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35657 } } peers { permanent_uuid: "2d9b46ff786c4955bc9afc8c51722f2e" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 46033 } } }
I20260807 18:17:47.892271 5081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d9b46ff786c4955bc9afc8c51722f2e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:17:47.892477 5108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5f4a971bf593423592de0955a984311c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4a971bf593423592de0955a984311c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38075 } } peers { permanent_uuid: "1fd3a75a6c974391b574eb8a31ed4837" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35657 } } peers { permanent_uuid: "2d9b46ff786c4955bc9afc8c51722f2e" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 46033 } } }
I20260807 18:17:47.892417 5106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fd3a75a6c974391b574eb8a31ed4837 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f4a971bf593423592de0955a984311c. Latest consensus state: current_term: 1 leader_uuid: "5f4a971bf593423592de0955a984311c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4a971bf593423592de0955a984311c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38075 } } peers { permanent_uuid: "1fd3a75a6c974391b574eb8a31ed4837" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35657 } } peers { permanent_uuid: "2d9b46ff786c4955bc9afc8c51722f2e" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 46033 } } }
I20260807 18:17:47.892789 5108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c [sys.catalog]: This master's current role is: LEADER
I20260807 18:17:47.892817 5106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fd3a75a6c974391b574eb8a31ed4837 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:17:47.895503 5130 mvcc.cc:205] Tried to move back new op lower bound from 7315974831273664512 to 7315974831109234688. Current Snapshot: MvccSnapshot[applied={T|T < 7315974831273664512}]
I20260807 18:17:47.900223 5144 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 18:17:47.914285 5106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fd3a75a6c974391b574eb8a31ed4837 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5f4a971bf593423592de0955a984311c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f4a971bf593423592de0955a984311c" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 38075 } } peers { permanent_uuid: "1fd3a75a6c974391b574eb8a31ed4837" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35657 } } peers { permanent_uuid: "2d9b46ff786c4955bc9afc8c51722f2e" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 46033 } } }
I20260807 18:17:47.914626 5106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fd3a75a6c974391b574eb8a31ed4837 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:17:47.927876 5117 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:17:47.930109 5117 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:17:47.953562 5117 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f4a971bf593423592de0955a984311c: Generated new TSK 0
I20260807 18:17:47.954873 5117 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:17:48.854480 5124 catalog_manager.cc:1259] Loaded cluster ID: 8dbf3a0830c34e02a2d177c19455604e
I20260807 18:17:48.854620 5124 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2d9b46ff786c4955bc9afc8c51722f2e: loading cluster ID for follower catalog manager: success
I20260807 18:17:48.857568 5124 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2d9b46ff786c4955bc9afc8c51722f2e: acquiring CA information for follower catalog manager: success
I20260807 18:17:48.859869 5124 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2d9b46ff786c4955bc9afc8c51722f2e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:17:48.905797 5144 catalog_manager.cc:1259] Loaded cluster ID: 8dbf3a0830c34e02a2d177c19455604e
I20260807 18:17:48.905965 5144 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1fd3a75a6c974391b574eb8a31ed4837: loading cluster ID for follower catalog manager: success
I20260807 18:17:48.910384 5144 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1fd3a75a6c974391b574eb8a31ed4837: acquiring CA information for follower catalog manager: success
I20260807 18:17:48.913650 5144 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1fd3a75a6c974391b574eb8a31ed4837: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:17:49.056061 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 4732
I20260807 18:17:49.057523 4817 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:49.176281 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 4732
W20260807 18:17:49.216584 4969 connection.cc:535] server connection from 127.0.0.1:44444 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 18:17:49.217427 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 4836
I20260807 18:17:49.218906 4930 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:49.345448 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 4836
I20260807 18:17:49.370321 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 4947
I20260807 18:17:49.371863 5035 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:49.486883 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 4947
2026-08-07T18:17:49Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5277 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:17:49 dist-test-slave-39l8 krb5kdc[5167](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:17:49 dist-test-slave-39l8 krb5kdc[5167](info): set up 2 sockets
Aug 07 18:17:49 dist-test-slave-39l8 krb5kdc[5167](info): commencing operation
krb5kdc: starting...
W20260807 18:17:50.843412 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.302s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:17:51 dist-test-slave-39l8 krb5kdc[5167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:17:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:17:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.110.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
I20260807 18:17:51.158097 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:46527
--webserver_interface=127.17.110.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:46311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:46527
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:17:51.524148 5185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:51.524554 5185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:51.524737 5185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:51.532406 5185 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:17:51.532593 5185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:51.532684 5185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:51.532791 5185 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:17:51.532868 5185 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:17:51.543466 5185 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:46311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:46527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:46527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:17:51.544925 5185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:51.546314 5185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:51.559345 5191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:51.559433 5192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:51.560158 5195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:51.560659 5185 server_base.cc:1034] running on GCE node
I20260807 18:17:51.561574 5185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:51.563436 5185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:51.564683 5185 hybrid_clock.cc:648] HybridClock initialized: now 1786126671564603 us; error 64 us; skew 500 ppm
I20260807 18:17:51.565099 5185 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:51 dist-test-slave-39l8 krb5kdc[5167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126671, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:51.574193 5185 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:17:51.576078 5185 webserver.cc:460] Webserver started at http://127.17.110.62:33733/ using document root <none> and password file <none>
I20260807 18:17:51.576905 5185 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:51.577040 5185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:51.577358 5185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:51.580938 5185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "64aa02a5537d4227898aef2abf58e976"
format_stamp: "Formatted at 2026-08-07 18:17:51 on dist-test-slave-39l8"
server_key: "8fe2e0c2dd53e7ce004dd073c0b5db95"
server_key_iv: "0ff176c144795d643ca67f5a675b2352"
server_key_version: "encryptionkey@0"
I20260807 18:17:51.581790 5185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "64aa02a5537d4227898aef2abf58e976"
format_stamp: "Formatted at 2026-08-07 18:17:51 on dist-test-slave-39l8"
server_key: "8fe2e0c2dd53e7ce004dd073c0b5db95"
server_key_iv: "0ff176c144795d643ca67f5a675b2352"
server_key_version: "encryptionkey@0"
I20260807 18:17:51.588120 5185 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 18:17:51.592327 5202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:51.593940 5185 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:17:51.594197 5185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "64aa02a5537d4227898aef2abf58e976"
format_stamp: "Formatted at 2026-08-07 18:17:51 on dist-test-slave-39l8"
server_key: "8fe2e0c2dd53e7ce004dd073c0b5db95"
server_key_iv: "0ff176c144795d643ca67f5a675b2352"
server_key_version: "encryptionkey@0"
I20260807 18:17:51.594451 5185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:51.619791 5185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:51.624938 5185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:51.625290 5185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:51.647380 5185 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:46527
I20260807 18:17:51.647401 5282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:46527 every 8 connection(s)
I20260807 18:17:51.649016 5185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:17:51.649782 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 5185
I20260807 18:17:51.650074 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:17:51.650987 17848 external_mini_cluster.cc:1428] Setting key a5c8cae8f779cde42a67fa59ea9ff1bf
I20260807 18:17:51.653900 5283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 18:17:51 dist-test-slave-39l8 krb5kdc[5167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:17:51.666671 5283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64aa02a5537d4227898aef2abf58e976: Bootstrap starting.
I20260807 18:17:51.670513 5283 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64aa02a5537d4227898aef2abf58e976: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:51.672057 5283 log.cc:824] T 00000000000000000000000000000000 P 64aa02a5537d4227898aef2abf58e976: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:51.676023 5283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64aa02a5537d4227898aef2abf58e976: No bootstrap required, opened a new log
I20260807 18:17:51.676443 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:51.653114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34036 (local address 127.17.110.62:46527)
0807 18:17:51.654012 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:51.654029 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:51.654083 (+ 54us) server_negotiation.cc:407] Connection header received
0807 18:17:51.655878 (+ 1795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:51.655902 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:51.656308 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:51.656730 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:51.658431 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:51.659889 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:51.661341 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:51.662053 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:51.662281 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:51.667564 (+ 5283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:51.667574 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:51.667592 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:51.667652 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:51.671119 (+ 3467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:51.672305 (+ 1186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:51.672314 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:51.672321 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:51.672583 (+ 262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:51.673120 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:51.673127 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:51.673131 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:51.673969 (+ 838us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:51.674196 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:51.674876 (+ 680us) server_negotiation.cc:299] Negotiation successful
0807 18:17:51.675299 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":396,"threads_started":1}
I20260807 18:17:51.681309 17848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:17:51.687909 5283 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64aa02a5537d4227898aef2abf58e976 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64aa02a5537d4227898aef2abf58e976" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 46527 } }
I20260807 18:17:51.688344 5283 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64aa02a5537d4227898aef2abf58e976 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:51.688436 5283 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64aa02a5537d4227898aef2abf58e976 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64aa02a5537d4227898aef2abf58e976, State: Initialized, Role: FOLLOWER
I20260807 18:17:51.689014 5283 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64aa02a5537d4227898aef2abf58e976 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64aa02a5537d4227898aef2abf58e976" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 46527 } }
I20260807 18:17:51.689244 5283 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64aa02a5537d4227898aef2abf58e976 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:17:51.689358 5283 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64aa02a5537d4227898aef2abf58e976 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:17:51.689505 5283 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64aa02a5537d4227898aef2abf58e976 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:51.692098 5283 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64aa02a5537d4227898aef2abf58e976 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64aa02a5537d4227898aef2abf58e976" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 46527 } }
I20260807 18:17:51.692664 5283 leader_election.cc:302] T 00000000000000000000000000000000 P 64aa02a5537d4227898aef2abf58e976 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64aa02a5537d4227898aef2abf58e976; no voters:
I20260807 18:17:51.693269 5283 leader_election.cc:288] T 00000000000000000000000000000000 P 64aa02a5537d4227898aef2abf58e976 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:17:51.693452 5290 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64aa02a5537d4227898aef2abf58e976 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:17:51.694296 5290 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64aa02a5537d4227898aef2abf58e976 [term 1 LEADER]: Becoming Leader. State: Replica: 64aa02a5537d4227898aef2abf58e976, State: Running, Role: LEADER
I20260807 18:17:51.695040 5290 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64aa02a5537d4227898aef2abf58e976 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64aa02a5537d4227898aef2abf58e976" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 46527 } }
I20260807 18:17:51.696067 5283 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64aa02a5537d4227898aef2abf58e976 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:17:51.699687 5291 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64aa02a5537d4227898aef2abf58e976 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64aa02a5537d4227898aef2abf58e976" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64aa02a5537d4227898aef2abf58e976" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 46527 } } }
I20260807 18:17:51.700310 5291 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64aa02a5537d4227898aef2abf58e976 [sys.catalog]: This master's current role is: LEADER
I20260807 18:17:51.699699 5294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64aa02a5537d4227898aef2abf58e976 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64aa02a5537d4227898aef2abf58e976. Latest consensus state: current_term: 1 leader_uuid: "64aa02a5537d4227898aef2abf58e976" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64aa02a5537d4227898aef2abf58e976" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 46527 } } }
I20260807 18:17:51.700637 5294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64aa02a5537d4227898aef2abf58e976 [sys.catalog]: This master's current role is: LEADER
I20260807 18:17:51.701524 5305 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:17:51.707530 5305 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:17:51.718335 5305 catalog_manager.cc:1347] Generated new cluster ID: b96c0ed684bd4988a1d36b9fe1bee9dc
I20260807 18:17:51.718503 5305 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:17:51.733661 5305 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:17:51.735328 5305 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:17:51.746908 5305 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64aa02a5537d4227898aef2abf58e976: Generated new TSK 0
I20260807 18:17:51.748126 5305 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:17:51.797317 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 5185
I20260807 18:17:51.799002 5277 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:51.901726 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 5185
2026-08-07T18:17:51Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2410 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:17:52 dist-test-slave-39l8 krb5kdc[5321](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:17:52 dist-test-slave-39l8 krb5kdc[5321](info): set up 2 sockets
Aug 07 18:17:52 dist-test-slave-39l8 krb5kdc[5321](info): commencing operation
krb5kdc: starting...
W20260807 18:17:53.261062 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.310s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:17:53 dist-test-slave-39l8 krb5kdc[5321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:17:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:17:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.110.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
I20260807 18:17:53.582486 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:42349
--webserver_interface=127.17.110.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:39359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:42349
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:17:53.955329 5338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:53.955745 5338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:53.955922 5338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:53.964071 5338 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:17:53.964284 5338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:53.964411 5338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:53.964516 5338 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:17:53.964602 5338 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:17:53.975502 5338 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:39359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:42349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:42349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:17:53.977033 5338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:53.978739 5338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:53.992507 5345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:53.992530 5347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:53.992507 5344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:53.993568 5338 server_base.cc:1034] running on GCE node
I20260807 18:17:53.994545 5338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:53.996368 5338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:53.997694 5338 hybrid_clock.cc:648] HybridClock initialized: now 1786126673997608 us; error 82 us; skew 500 ppm
I20260807 18:17:53.998180 5338 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:53 dist-test-slave-39l8 krb5kdc[5321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126673, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:54.002698 5338 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:17:54.005718 5338 webserver.cc:460] Webserver started at http://127.17.110.62:38579/ using document root <none> and password file <none>
I20260807 18:17:54.006639 5338 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:54.006888 5338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:54.007395 5338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:54.011312 5338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "c9a45c2d73e04aeca5f641cd1d765906"
format_stamp: "Formatted at 2026-08-07 18:17:54 on dist-test-slave-39l8"
server_key: "db76b691220b44585db6aef8f9bb8743"
server_key_iv: "573da03a6a251cfc5a44111b20a339a6"
server_key_version: "encryptionkey@0"
I20260807 18:17:54.012352 5338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "c9a45c2d73e04aeca5f641cd1d765906"
format_stamp: "Formatted at 2026-08-07 18:17:54 on dist-test-slave-39l8"
server_key: "db76b691220b44585db6aef8f9bb8743"
server_key_iv: "573da03a6a251cfc5a44111b20a339a6"
server_key_version: "encryptionkey@0"
I20260807 18:17:54.020283 5338 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260807 18:17:54.025913 5358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:54.027879 5338 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 18:17:54.028152 5338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "c9a45c2d73e04aeca5f641cd1d765906"
format_stamp: "Formatted at 2026-08-07 18:17:54 on dist-test-slave-39l8"
server_key: "db76b691220b44585db6aef8f9bb8743"
server_key_iv: "573da03a6a251cfc5a44111b20a339a6"
server_key_version: "encryptionkey@0"
I20260807 18:17:54.028429 5338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:54.051841 5338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:54.056627 5338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:54.056965 5338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:54.078683 5338 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:42349
I20260807 18:17:54.078750 5439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:42349 every 8 connection(s)
I20260807 18:17:54.080305 5338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:17:54.084614 5441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:54.085733 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 5338
I20260807 18:17:54.085985 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:17:54.086984 17848 external_mini_cluster.cc:1428] Setting key f15c9cbb08216e72779c84d2d391ad69
I20260807 18:17:54.101722 5441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906: Bootstrap starting.
Aug 07 18:17:54 dist-test-slave-39l8 krb5kdc[5321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:17:54.106300 5441 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:54.107995 5441 log.cc:824] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:54.113133 5441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906: No bootstrap required, opened a new log
I20260807 18:17:54.117280 5444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:54.089750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47406 (local address 127.17.110.62:42349)
0807 18:17:54.090857 (+ 1107us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:54.090876 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:54.090929 (+ 53us) server_negotiation.cc:407] Connection header received
0807 18:17:54.092875 (+ 1946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:54.092914 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:54.093469 (+ 555us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:54.093900 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:54.096084 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:54.097666 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:54.099784 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:54.100771 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:54.100965 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:54.106882 (+ 5917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:54.106904 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:54.106931 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:54.107005 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:54.112176 (+ 5171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:54.113204 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:54.113237 (+ 33us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:54.113250 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:54.113490 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:54.113959 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:54.113965 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:54.113968 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:54.114609 (+ 641us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:54.115054 (+ 445us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:54.115567 (+ 513us) server_negotiation.cc:299] Negotiation successful
0807 18:17:54.116105 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":745,"thread_start_us":488,"threads_started":1}
I20260807 18:17:54.123361 5441 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9a45c2d73e04aeca5f641cd1d765906" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 42349 } }
I20260807 18:17:54.123854 5441 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:54.124002 5441 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9a45c2d73e04aeca5f641cd1d765906, State: Initialized, Role: FOLLOWER
I20260807 18:17:54.124686 5441 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9a45c2d73e04aeca5f641cd1d765906" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 42349 } }
I20260807 18:17:54.124962 5441 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:17:54.125162 5441 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:17:54.125367 5441 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:54.128113 5441 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9a45c2d73e04aeca5f641cd1d765906" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 42349 } }
I20260807 18:17:54.128736 5441 leader_election.cc:302] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9a45c2d73e04aeca5f641cd1d765906; no voters:
I20260807 18:17:54.129470 5441 leader_election.cc:288] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:17:54.129652 5447 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:17:54.130439 5447 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [term 1 LEADER]: Becoming Leader. State: Replica: c9a45c2d73e04aeca5f641cd1d765906, State: Running, Role: LEADER
I20260807 18:17:54.131204 5447 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9a45c2d73e04aeca5f641cd1d765906" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 42349 } }
I20260807 18:17:54.132115 5441 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:17:54.134816 5449 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9a45c2d73e04aeca5f641cd1d765906" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9a45c2d73e04aeca5f641cd1d765906" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 42349 } } }
I20260807 18:17:54.134764 5450 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9a45c2d73e04aeca5f641cd1d765906. Latest consensus state: current_term: 1 leader_uuid: "c9a45c2d73e04aeca5f641cd1d765906" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9a45c2d73e04aeca5f641cd1d765906" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 42349 } } }
I20260807 18:17:54.135416 5449 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [sys.catalog]: This master's current role is: LEADER
I20260807 18:17:54.135428 5450 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [sys.catalog]: This master's current role is: LEADER
I20260807 18:17:54.137840 5457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:17:54.144483 5457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:17:54.156404 5457 catalog_manager.cc:1347] Generated new cluster ID: 2d92dac564b8453880b170c5b3a209fa
I20260807 18:17:54.156622 5457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:17:54.202620 5457 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:17:54.204842 5457 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:17:54.228003 5457 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906: Generated new TSK 0
I20260807 18:17:54.229200 5457 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.110.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
I20260807 18:17:54.323661 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.1:0
--local_ip_for_outbound_sockets=127.17.110.1
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:42349
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:39359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:17:54.698244 5475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:54.698639 5475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:54.698846 5475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:54.706555 5475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:54.706785 5475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:54.706981 5475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.1
I20260807 18:17:54.717486 5475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:39359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:42349
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.1
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:17:54.719075 5475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:54.720522 5475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:54.735363 5481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:54.735421 5486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:54.735384 5484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:54.737212 5475 server_base.cc:1034] running on GCE node
I20260807 18:17:54.738065 5475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:54.739312 5475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:54.740628 5475 hybrid_clock.cc:648] HybridClock initialized: now 1786126674740594 us; error 75 us; skew 500 ppm
I20260807 18:17:54.741000 5475 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:54 dist-test-slave-39l8 krb5kdc[5321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126674, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:54.744820 5475 init.cc:377] Logged in from keytab as kudu/127.17.110.1@KRBTEST.COM (short username kudu)
I20260807 18:17:54.746579 5475 webserver.cc:460] Webserver started at http://127.17.110.1:44317/ using document root <none> and password file <none>
I20260807 18:17:54.747421 5475 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:54.747543 5475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:54.747820 5475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:54.751264 5475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/data/instance:
uuid: "aeb211ad2c004cd0a3a40e510f67ac98"
format_stamp: "Formatted at 2026-08-07 18:17:54 on dist-test-slave-39l8"
server_key: "a931ebb27a09180b4bc7c57e28a17834"
server_key_iv: "f602632313b312703a42ac4e7cda1a48"
server_key_version: "encryptionkey@0"
I20260807 18:17:54.752229 5475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance:
uuid: "aeb211ad2c004cd0a3a40e510f67ac98"
format_stamp: "Formatted at 2026-08-07 18:17:54 on dist-test-slave-39l8"
server_key: "a931ebb27a09180b4bc7c57e28a17834"
server_key_iv: "f602632313b312703a42ac4e7cda1a48"
server_key_version: "encryptionkey@0"
I20260807 18:17:54.758989 5475 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 18:17:54.763228 5496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:54.764983 5475 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:17:54.765302 5475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/wal
uuid: "aeb211ad2c004cd0a3a40e510f67ac98"
format_stamp: "Formatted at 2026-08-07 18:17:54 on dist-test-slave-39l8"
server_key: "a931ebb27a09180b4bc7c57e28a17834"
server_key_iv: "f602632313b312703a42ac4e7cda1a48"
server_key_version: "encryptionkey@0"
I20260807 18:17:54.765581 5475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:54.790375 5475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:54.795977 5475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:54.796344 5475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:54.797538 5475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:17:54.799283 5475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:17:54.799369 5475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:54.799501 5475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:17:54.799572 5475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:54.836809 5475 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.1:44935
I20260807 18:17:54.836911 5660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.1:44935 every 8 connection(s)
I20260807 18:17:54.838224 5475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
I20260807 18:17:54.839515 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 5475
I20260807 18:17:54.839766 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance
I20260807 18:17:54.840595 17848 external_mini_cluster.cc:1428] Setting key 831bc1985023322161edef54028b521e
Aug 07 18:17:54 dist-test-slave-39l8 krb5kdc[5321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126674, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:17:54.863822 5664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:54.841523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:46337 (local address 127.17.110.62:42349)
0807 18:17:54.842009 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:54.842014 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:54.843070 (+ 1056us) server_negotiation.cc:407] Connection header received
0807 18:17:54.845174 (+ 2104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:54.845180 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:54.845281 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:54.845419 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:54.847232 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:54.848011 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:54.849832 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:54.850308 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:54.850466 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:54.855374 (+ 4908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:54.855381 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:54.855385 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:54.855434 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:54.858514 (+ 3080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:54.860007 (+ 1493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:54.860014 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:54.860016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:54.860222 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:54.861036 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:54.861042 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:54.861044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:54.861376 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:54.861518 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:54.862944 (+ 1426us) server_negotiation.cc:299] Negotiation successful
0807 18:17:54.863505 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":249,"threads_started":1}
I20260807 18:17:54.865654 5665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:54.841987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.1:46337)
0807 18:17:54.842862 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:54.842921 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:54.844713 (+ 1792us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:54.845697 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:54.845707 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:54.846282 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:54.847025 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:54.847054 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:54.848219 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:54.848224 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:54.849632 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:54.849648 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:54.850593 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:54.850602 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:54.850950 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:54.851791 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:17:54.851819 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:54.854927 (+ 3108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:54.858815 (+ 3888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:54.858831 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:54.858857 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:54.859754 (+ 897us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:54.860422 (+ 668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:54.860431 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:54.860438 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:54.860802 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:54.861558 (+ 756us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:54.861571 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:54.862155 (+ 584us) client_negotiation.cc:769] Sending connection context
0807 18:17:54.862587 (+ 432us) client_negotiation.cc:240] Negotiation successful
0807 18:17:54.863609 (+ 1022us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":319,"threads_started":1}
I20260807 18:17:54.868607 5662 heartbeater.cc:347] Connected to a master server at 127.17.110.62:42349
I20260807 18:17:54.869036 5662 heartbeater.cc:464] Registering TS with master...
I20260807 18:17:54.870216 5662 heartbeater.cc:511] Master 127.17.110.62:42349 requested a full tablet report, sending...
I20260807 18:17:54.873420 5384 ts_manager.cc:194] Registered new tserver with Master: aeb211ad2c004cd0a3a40e510f67ac98 (127.17.110.1:44935)
I20260807 18:17:54.875955 5384 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.1@KRBTEST.COM'} at 127.17.110.1:46337
WARNING: no policy specified for kudu/127.17.110.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
I20260807 18:17:54.924399 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.2:0
--local_ip_for_outbound_sockets=127.17.110.2
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:42349
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:39359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:17:55.319620 5671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:55.319993 5671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:55.320175 5671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:55.327883 5671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:55.328078 5671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:55.328315 5671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.2
I20260807 18:17:55.338845 5671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:39359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:42349
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.2
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:17:55.340431 5671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:55.341856 5671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:55.357370 5681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:55.357347 5677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:55.357553 5683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:55.358338 5671 server_base.cc:1034] running on GCE node
I20260807 18:17:55.359202 5671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:55.360373 5671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:55.361676 5671 hybrid_clock.cc:648] HybridClock initialized: now 1786126675361617 us; error 57 us; skew 500 ppm
I20260807 18:17:55.362171 5671 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:55 dist-test-slave-39l8 krb5kdc[5321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126675, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:55.366406 5671 init.cc:377] Logged in from keytab as kudu/127.17.110.2@KRBTEST.COM (short username kudu)
I20260807 18:17:55.368455 5671 webserver.cc:460] Webserver started at http://127.17.110.2:36373/ using document root <none> and password file <none>
I20260807 18:17:55.369369 5671 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:55.369541 5671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:55.369890 5671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:55.373387 5671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/data/instance:
uuid: "f9819a0a0990421994d36137ef0bf904"
format_stamp: "Formatted at 2026-08-07 18:17:55 on dist-test-slave-39l8"
server_key: "8840004cfcbfe9919dab3cf453acf8e4"
server_key_iv: "9e5d3a254e8862dfe80a99555d35578d"
server_key_version: "encryptionkey@0"
I20260807 18:17:55.374353 5671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance:
uuid: "f9819a0a0990421994d36137ef0bf904"
format_stamp: "Formatted at 2026-08-07 18:17:55 on dist-test-slave-39l8"
server_key: "8840004cfcbfe9919dab3cf453acf8e4"
server_key_iv: "9e5d3a254e8862dfe80a99555d35578d"
server_key_version: "encryptionkey@0"
I20260807 18:17:55.381455 5671 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260807 18:17:55.385881 5696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:55.387814 5671 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 18:17:55.388100 5671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/wal
uuid: "f9819a0a0990421994d36137ef0bf904"
format_stamp: "Formatted at 2026-08-07 18:17:55 on dist-test-slave-39l8"
server_key: "8840004cfcbfe9919dab3cf453acf8e4"
server_key_iv: "9e5d3a254e8862dfe80a99555d35578d"
server_key_version: "encryptionkey@0"
I20260807 18:17:55.388382 5671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:55.428517 5671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:55.434345 5671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:55.434736 5671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:55.435873 5671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:17:55.437552 5671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:17:55.437672 5671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:55.437865 5671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:17:55.437923 5671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:55.477969 5671 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.2:33801
I20260807 18:17:55.478046 5859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.2:33801 every 8 connection(s)
I20260807 18:17:55.479625 5671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
I20260807 18:17:55.489209 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 5671
I20260807 18:17:55.489431 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance
I20260807 18:17:55.489984 17848 external_mini_cluster.cc:1428] Setting key a26a2a66d695c3bbb78116de7986d2ce
Aug 07 18:17:55 dist-test-slave-39l8 krb5kdc[5321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126675, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:17:55.505734 5865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:55.483024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:34405 (local address 127.17.110.62:42349)
0807 18:17:55.483585 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:55.483590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:55.484697 (+ 1107us) server_negotiation.cc:407] Connection header received
0807 18:17:55.486927 (+ 2230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:55.486934 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:55.487050 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:55.487191 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:55.489240 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:55.490301 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:55.492825 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:55.493423 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:55.493558 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:55.498994 (+ 5436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:55.499001 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:55.499005 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:55.499093 (+ 88us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:55.502025 (+ 2932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:55.503109 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:55.503113 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:55.503116 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:55.503255 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:55.503768 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:55.503772 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:55.503774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:55.504081 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:55.504253 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:55.505065 (+ 812us) server_negotiation.cc:299] Negotiation successful
0807 18:17:55.505333 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":299,"threads_started":1}
I20260807 18:17:55.506945 5866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:55.483556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.2:34405)
0807 18:17:55.484469 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:55.484528 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:55.486525 (+ 1997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:55.487484 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:55.487495 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:55.488119 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:55.488907 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:55.488942 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:55.490525 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:55.490532 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:55.492605 (+ 2073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:55.492632 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:55.493735 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:55.493747 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:55.494211 (+ 464us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:55.495227 (+ 1016us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:17:55.495278 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:55.498588 (+ 3310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:55.502249 (+ 3661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:55.502260 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:55.502306 (+ 46us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:55.502931 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:55.503388 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:55.503393 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:55.503397 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:55.503623 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:55.504224 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:55.504234 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:55.504648 (+ 414us) client_negotiation.cc:769] Sending connection context
0807 18:17:55.504955 (+ 307us) client_negotiation.cc:240] Negotiation successful
0807 18:17:55.505584 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":446,"threads_started":1}
I20260807 18:17:55.508924 5861 heartbeater.cc:347] Connected to a master server at 127.17.110.62:42349
I20260807 18:17:55.509366 5861 heartbeater.cc:464] Registering TS with master...
I20260807 18:17:55.510310 5861 heartbeater.cc:511] Master 127.17.110.62:42349 requested a full tablet report, sending...
I20260807 18:17:55.512328 5384 ts_manager.cc:194] Registered new tserver with Master: f9819a0a0990421994d36137ef0bf904 (127.17.110.2:33801)
I20260807 18:17:55.513273 5384 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.2@KRBTEST.COM'} at 127.17.110.2:34405
WARNING: no policy specified for kudu/127.17.110.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
I20260807 18:17:55.562994 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.3:0
--local_ip_for_outbound_sockets=127.17.110.3
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:42349
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:39359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:17:55.881068 5662 heartbeater.cc:503] Master 127.17.110.62:42349 was elected leader, sending a full tablet report...
W20260807 18:17:55.926810 5872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:55.927153 5872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:55.927259 5872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:55.935245 5872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:55.935376 5872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:55.935495 5872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.3
I20260807 18:17:55.945962 5872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:39359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:42349
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.3
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:17:55.947346 5872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:55.948743 5872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:55.963380 5879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:55.963652 5881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:55.963435 5878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:55.964406 5872 server_base.cc:1034] running on GCE node
I20260807 18:17:55.965140 5872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:55.966274 5872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:55.967531 5872 hybrid_clock.cc:648] HybridClock initialized: now 1786126675967467 us; error 55 us; skew 500 ppm
I20260807 18:17:55.967904 5872 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:55 dist-test-slave-39l8 krb5kdc[5321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126675, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:55.971864 5872 init.cc:377] Logged in from keytab as kudu/127.17.110.3@KRBTEST.COM (short username kudu)
I20260807 18:17:55.973604 5872 webserver.cc:460] Webserver started at http://127.17.110.3:44603/ using document root <none> and password file <none>
I20260807 18:17:55.974416 5872 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:55.974586 5872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:55.974987 5872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:55.978252 5872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/data/instance:
uuid: "c8cf58c0a4ff4d22933a471b69be07e9"
format_stamp: "Formatted at 2026-08-07 18:17:55 on dist-test-slave-39l8"
server_key: "4a892bbf4046e5805f4eaa8133c2f917"
server_key_iv: "e389a16ce06e2a341ee4370d96a3cd3e"
server_key_version: "encryptionkey@0"
I20260807 18:17:55.979195 5872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance:
uuid: "c8cf58c0a4ff4d22933a471b69be07e9"
format_stamp: "Formatted at 2026-08-07 18:17:55 on dist-test-slave-39l8"
server_key: "4a892bbf4046e5805f4eaa8133c2f917"
server_key_iv: "e389a16ce06e2a341ee4370d96a3cd3e"
server_key_version: "encryptionkey@0"
I20260807 18:17:55.985309 5872 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 18:17:55.989447 5890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:55.991161 5872 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 18:17:55.991375 5872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/wal
uuid: "c8cf58c0a4ff4d22933a471b69be07e9"
format_stamp: "Formatted at 2026-08-07 18:17:55 on dist-test-slave-39l8"
server_key: "4a892bbf4046e5805f4eaa8133c2f917"
server_key_iv: "e389a16ce06e2a341ee4370d96a3cd3e"
server_key_version: "encryptionkey@0"
I20260807 18:17:55.991632 5872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:56.011262 5872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:56.018265 5872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:56.018755 5872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:56.019927 5872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:17:56.021447 5872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:17:56.021528 5872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:56.021628 5872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:17:56.021682 5872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:56.057250 5872 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.3:38927
I20260807 18:17:56.057286 6045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.3:38927 every 8 connection(s)
I20260807 18:17:56.058758 5872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
I20260807 18:17:56.066025 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 5872
I20260807 18:17:56.066355 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance
I20260807 18:17:56.066954 17848 external_mini_cluster.cc:1428] Setting key 60a301956a6ccfaa756480ab19e8d33d
Aug 07 18:17:56 dist-test-slave-39l8 krb5kdc[5321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126675, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:17:56.085155 6052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:56.061997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:37135 (local address 127.17.110.62:42349)
0807 18:17:56.062544 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:56.062549 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:56.064004 (+ 1455us) server_negotiation.cc:407] Connection header received
0807 18:17:56.066020 (+ 2016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:56.066027 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:56.066139 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:56.066301 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:56.068677 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.069920 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.072311 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.072752 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.072872 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:56.077135 (+ 4263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:56.077202 (+ 67us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:56.077206 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:56.077249 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:56.080413 (+ 3164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:56.081632 (+ 1219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:56.081639 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:56.081643 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:56.081854 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:56.082468 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:56.082473 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:56.082477 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:56.083023 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:56.083280 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:56.084371 (+ 1091us) server_negotiation.cc:299] Negotiation successful
0807 18:17:56.084653 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":286,"threads_started":1}
I20260807 18:17:56.086287 6053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:56.062767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.3:37135)
0807 18:17:56.063758 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:56.063832 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:56.065643 (+ 1811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:56.066694 (+ 1051us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:56.066755 (+ 61us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:56.067481 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:56.068407 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:56.068441 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.070123 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.070129 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:56.072060 (+ 1931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:56.072078 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.073002 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.073008 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:56.073282 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:56.074018 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:17:56.074044 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:56.076810 (+ 2766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:56.080616 (+ 3806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:56.080629 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:56.080659 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:56.081363 (+ 704us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:56.082024 (+ 661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:56.082029 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:56.082034 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:56.082318 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:56.083246 (+ 928us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:56.083257 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:56.083823 (+ 566us) client_negotiation.cc:769] Sending connection context
0807 18:17:56.084251 (+ 428us) client_negotiation.cc:240] Negotiation successful
0807 18:17:56.084906 (+ 655us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":354,"threads_started":1}
I20260807 18:17:56.088004 6047 heartbeater.cc:347] Connected to a master server at 127.17.110.62:42349
I20260807 18:17:56.088435 6047 heartbeater.cc:464] Registering TS with master...
I20260807 18:17:56.089346 6047 heartbeater.cc:511] Master 127.17.110.62:42349 requested a full tablet report, sending...
I20260807 18:17:56.091244 5384 ts_manager.cc:194] Registered new tserver with Master: c8cf58c0a4ff4d22933a471b69be07e9 (127.17.110.3:38927)
I20260807 18:17:56.092234 5384 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.3@KRBTEST.COM'} at 127.17.110.3:37135
I20260807 18:17:56.097458 17848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:17:56.118494 6052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:56.103136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47414 (local address 127.17.110.62:42349)
0807 18:17:56.103348 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:56.103353 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:56.103767 (+ 414us) server_negotiation.cc:407] Connection header received
0807 18:17:56.104013 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:56.104018 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:56.104140 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:56.104252 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:56.105883 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.106857 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.108452 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.108908 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.109053 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:56.112259 (+ 3206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:56.112265 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:56.112269 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:56.112317 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:56.115411 (+ 3094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:56.116331 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:56.116336 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:56.116339 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:56.116500 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:56.116918 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:56.116922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:56.116924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:56.117347 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:56.117561 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:56.117943 (+ 382us) server_negotiation.cc:299] Negotiation successful
0807 18:17:56.118204 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 18:17:56.128643 5384 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47414:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:17:56.136909 5384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:17:56.170436 6065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:56.151283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:44935 (local address 127.0.0.1:50670)
0807 18:17:56.153323 (+ 2040us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:56.153375 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:56.153565 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:56.156409 (+ 2844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:56.156420 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:56.156475 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:56.157021 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:56.157038 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.161477 (+ 4439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.161512 (+ 35us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:56.166444 (+ 4932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:56.166460 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.169043 (+ 2583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.169050 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:56.169436 (+ 386us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:56.169573 (+ 137us) client_negotiation.cc:769] Sending connection context
0807 18:17:56.169719 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 18:17:56.169987 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1794,"thread_start_us":1645,"threads_started":1}
I20260807 18:17:56.171375 6066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:56.151630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50670 (local address 127.17.110.1:44935)
0807 18:17:56.152738 (+ 1108us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:56.152749 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:56.153783 (+ 1034us) server_negotiation.cc:407] Connection header received
0807 18:17:56.153913 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:56.153923 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:56.155755 (+ 1832us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:56.156030 (+ 275us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:56.157334 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.161083 (+ 3749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.166782 (+ 5699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.168474 (+ 1692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.168624 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:56.168881 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:56.170756 (+ 1875us) server_negotiation.cc:299] Negotiation successful
0807 18:17:56.170982 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":866,"thread_start_us":652,"threads_started":1}
I20260807 18:17:56.174674 6070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:56.154115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44980 (local address 127.17.110.3:38927)
0807 18:17:56.166344 (+ 12229us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:56.166360 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:56.166392 (+ 32us) server_negotiation.cc:407] Connection header received
0807 18:17:56.166483 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:56.166490 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:56.166835 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:56.167134 (+ 299us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:56.167802 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.169418 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.172334 (+ 2916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.173333 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.173502 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:56.173786 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:56.174149 (+ 363us) server_negotiation.cc:299] Negotiation successful
0807 18:17:56.174329 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12031,"thread_start_us":11886,"threads_started":1}
I20260807 18:17:56.174757 6071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:56.154548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:38927 (local address 127.0.0.1:44980)
0807 18:17:56.165838 (+ 11290us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:56.165861 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:56.165988 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:56.167115 (+ 1127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:56.167120 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:56.167152 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:56.167550 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:56.167563 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.169685 (+ 2122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.169694 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:56.172138 (+ 2444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:56.172152 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.173585 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.173592 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:56.173936 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:56.173964 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 18:17:56.174088 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 18:17:56.174243 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11124,"mutex_wait_us":76,"thread_start_us":10951,"threads_started":1}
I20260807 18:17:56.175853 5566 tablet_service.cc:1467] Processing CreateTablet for tablet c543aee1eeed4714a0f8362effee8de7 (DEFAULT_TABLE table=test-table [id=d5717beeb0824c60b424819696be133c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:17:56.177380 6067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:56.152547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:33801 (local address 127.0.0.1:41862)
0807 18:17:56.164935 (+ 12388us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:56.164965 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:56.165179 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:56.168798 (+ 3619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:56.168803 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:56.168838 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:56.169355 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:56.169368 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.171230 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.171234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:56.174405 (+ 3171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:56.174421 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.176412 (+ 1991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.176420 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:56.176609 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:56.176644 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 18:17:56.176761 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 18:17:56.176922 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12167,"mutex_wait_us":33,"thread_start_us":13545,"threads_started":1}
I20260807 18:17:56.178181 5566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c543aee1eeed4714a0f8362effee8de7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:56.178265 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:56.152963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41862 (local address 127.17.110.2:33801)
0807 18:17:56.167945 (+ 14982us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:56.167951 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:56.167978 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:17:56.168068 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:56.168074 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:56.168293 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:56.168518 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:56.169652 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.171054 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.174638 (+ 3584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.176152 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.176310 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:56.176524 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:56.177665 (+ 1141us) server_negotiation.cc:299] Negotiation successful
0807 18:17:56.177866 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14823,"thread_start_us":14670,"threads_started":1}
I20260807 18:17:56.179340 5960 tablet_service.cc:1467] Processing CreateTablet for tablet c543aee1eeed4714a0f8362effee8de7 (DEFAULT_TABLE table=test-table [id=d5717beeb0824c60b424819696be133c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:17:56.181099 5960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c543aee1eeed4714a0f8362effee8de7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:56.182642 5766 tablet_service.cc:1467] Processing CreateTablet for tablet c543aee1eeed4714a0f8362effee8de7 (DEFAULT_TABLE table=test-table [id=d5717beeb0824c60b424819696be133c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:17:56.184352 5766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c543aee1eeed4714a0f8362effee8de7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:56.190876 6072 tablet_bootstrap.cc:492] T c543aee1eeed4714a0f8362effee8de7 P aeb211ad2c004cd0a3a40e510f67ac98: Bootstrap starting.
I20260807 18:17:56.195488 6072 tablet_bootstrap.cc:654] T c543aee1eeed4714a0f8362effee8de7 P aeb211ad2c004cd0a3a40e510f67ac98: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:56.196744 6072 log.cc:824] T c543aee1eeed4714a0f8362effee8de7 P aeb211ad2c004cd0a3a40e510f67ac98: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:56.198076 6075 tablet_bootstrap.cc:492] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9: Bootstrap starting.
I20260807 18:17:56.201005 6077 tablet_bootstrap.cc:492] T c543aee1eeed4714a0f8362effee8de7 P f9819a0a0990421994d36137ef0bf904: Bootstrap starting.
I20260807 18:17:56.203616 6075 tablet_bootstrap.cc:654] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:56.204691 6077 tablet_bootstrap.cc:654] T c543aee1eeed4714a0f8362effee8de7 P f9819a0a0990421994d36137ef0bf904: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:56.205153 6075 log.cc:824] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:56.205952 6077 log.cc:824] T c543aee1eeed4714a0f8362effee8de7 P f9819a0a0990421994d36137ef0bf904: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:56.206626 6072 tablet_bootstrap.cc:492] T c543aee1eeed4714a0f8362effee8de7 P aeb211ad2c004cd0a3a40e510f67ac98: No bootstrap required, opened a new log
I20260807 18:17:56.207067 6072 ts_tablet_manager.cc:1397] T c543aee1eeed4714a0f8362effee8de7 P aeb211ad2c004cd0a3a40e510f67ac98: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.004s
I20260807 18:17:56.208854 6075 tablet_bootstrap.cc:492] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9: No bootstrap required, opened a new log
I20260807 18:17:56.209153 6075 ts_tablet_manager.cc:1397] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260807 18:17:56.211434 6077 tablet_bootstrap.cc:492] T c543aee1eeed4714a0f8362effee8de7 P f9819a0a0990421994d36137ef0bf904: No bootstrap required, opened a new log
I20260807 18:17:56.211755 6077 ts_tablet_manager.cc:1397] T c543aee1eeed4714a0f8362effee8de7 P f9819a0a0990421994d36137ef0bf904: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.004s
I20260807 18:17:56.218597 6072 raft_consensus.cc:348] T c543aee1eeed4714a0f8362effee8de7 P aeb211ad2c004cd0a3a40e510f67ac98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeb211ad2c004cd0a3a40e510f67ac98" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 44935 } } peers { permanent_uuid: "c8cf58c0a4ff4d22933a471b69be07e9" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 38927 } } peers { permanent_uuid: "f9819a0a0990421994d36137ef0bf904" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 33801 } }
I20260807 18:17:56.219187 6072 raft_consensus.cc:374] T c543aee1eeed4714a0f8362effee8de7 P aeb211ad2c004cd0a3a40e510f67ac98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:56.219376 6072 raft_consensus.cc:729] T c543aee1eeed4714a0f8362effee8de7 P aeb211ad2c004cd0a3a40e510f67ac98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aeb211ad2c004cd0a3a40e510f67ac98, State: Initialized, Role: FOLLOWER
I20260807 18:17:56.220152 6072 consensus_queue.cc:260] T c543aee1eeed4714a0f8362effee8de7 P aeb211ad2c004cd0a3a40e510f67ac98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeb211ad2c004cd0a3a40e510f67ac98" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 44935 } } peers { permanent_uuid: "c8cf58c0a4ff4d22933a471b69be07e9" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 38927 } } peers { permanent_uuid: "f9819a0a0990421994d36137ef0bf904" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 33801 } }
I20260807 18:17:56.220153 6077 raft_consensus.cc:348] T c543aee1eeed4714a0f8362effee8de7 P f9819a0a0990421994d36137ef0bf904 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeb211ad2c004cd0a3a40e510f67ac98" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 44935 } } peers { permanent_uuid: "c8cf58c0a4ff4d22933a471b69be07e9" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 38927 } } peers { permanent_uuid: "f9819a0a0990421994d36137ef0bf904" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 33801 } }
I20260807 18:17:56.220544 6077 raft_consensus.cc:374] T c543aee1eeed4714a0f8362effee8de7 P f9819a0a0990421994d36137ef0bf904 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:56.220326 6075 raft_consensus.cc:348] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeb211ad2c004cd0a3a40e510f67ac98" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 44935 } } peers { permanent_uuid: "c8cf58c0a4ff4d22933a471b69be07e9" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 38927 } } peers { permanent_uuid: "f9819a0a0990421994d36137ef0bf904" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 33801 } }
I20260807 18:17:56.220711 6077 raft_consensus.cc:729] T c543aee1eeed4714a0f8362effee8de7 P f9819a0a0990421994d36137ef0bf904 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9819a0a0990421994d36137ef0bf904, State: Initialized, Role: FOLLOWER
I20260807 18:17:56.220934 6075 raft_consensus.cc:374] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:56.221215 6075 raft_consensus.cc:729] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8cf58c0a4ff4d22933a471b69be07e9, State: Initialized, Role: FOLLOWER
I20260807 18:17:56.221423 6077 consensus_queue.cc:260] T c543aee1eeed4714a0f8362effee8de7 P f9819a0a0990421994d36137ef0bf904 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeb211ad2c004cd0a3a40e510f67ac98" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 44935 } } peers { permanent_uuid: "c8cf58c0a4ff4d22933a471b69be07e9" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 38927 } } peers { permanent_uuid: "f9819a0a0990421994d36137ef0bf904" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 33801 } }
I20260807 18:17:56.221988 6075 consensus_queue.cc:260] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeb211ad2c004cd0a3a40e510f67ac98" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 44935 } } peers { permanent_uuid: "c8cf58c0a4ff4d22933a471b69be07e9" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 38927 } } peers { permanent_uuid: "f9819a0a0990421994d36137ef0bf904" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 33801 } }
I20260807 18:17:56.222277 6072 ts_tablet_manager.cc:1428] T c543aee1eeed4714a0f8362effee8de7 P aeb211ad2c004cd0a3a40e510f67ac98: Time spent starting tablet: real 0.015s user 0.010s sys 0.005s
I20260807 18:17:56.223635 6047 heartbeater.cc:503] Master 127.17.110.62:42349 was elected leader, sending a full tablet report...
I20260807 18:17:56.224081 6075 ts_tablet_manager.cc:1428] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260807 18:17:56.225026 5861 heartbeater.cc:503] Master 127.17.110.62:42349 was elected leader, sending a full tablet report...
I20260807 18:17:56.225804 6077 ts_tablet_manager.cc:1428] T c543aee1eeed4714a0f8362effee8de7 P f9819a0a0990421994d36137ef0bf904: Time spent starting tablet: real 0.014s user 0.010s sys 0.003s
W20260807 18:17:56.231982 5863 tablet.cc:2369] T c543aee1eeed4714a0f8362effee8de7 P f9819a0a0990421994d36137ef0bf904: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:17:56.310679 6051 tablet.cc:2369] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:17:56.341238 5663 tablet.cc:2369] T c543aee1eeed4714a0f8362effee8de7 P aeb211ad2c004cd0a3a40e510f67ac98: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:17:56.372406 6085 raft_consensus.cc:482] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:17:56.372820 6085 raft_consensus.cc:504] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeb211ad2c004cd0a3a40e510f67ac98" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 44935 } } peers { permanent_uuid: "c8cf58c0a4ff4d22933a471b69be07e9" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 38927 } } peers { permanent_uuid: "f9819a0a0990421994d36137ef0bf904" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 33801 } }
I20260807 18:17:56.374836 6085 leader_election.cc:288] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aeb211ad2c004cd0a3a40e510f67ac98 (127.17.110.1:44935), f9819a0a0990421994d36137ef0bf904 (127.17.110.2:33801)
I20260807 18:17:56.381558 6066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:56.375259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:55627 (local address 127.17.110.1:44935)
0807 18:17:56.375493 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:56.375501 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:56.375662 (+ 161us) server_negotiation.cc:407] Connection header received
0807 18:17:56.376072 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:56.376081 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:56.376188 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:56.376307 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:56.377064 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.377841 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.379743 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.380599 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.380785 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:56.380869 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:56.381164 (+ 295us) server_negotiation.cc:299] Negotiation successful
0807 18:17:56.381300 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:17:56.381603 6053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:56.375186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:44935 (local address 127.17.110.3:55627)
0807 18:17:56.375531 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:56.375559 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:56.375742 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:56.376334 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:56.376339 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:56.376369 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:56.376599 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:56.376764 (+ 165us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.378000 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.378006 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:56.379557 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:56.379576 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.380776 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.380782 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:56.380978 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:56.381007 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 18:17:56.381123 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 18:17:56.381276 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":1,"spinlock_wait_cycles":38656}
I20260807 18:17:56.381804 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:56.375698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:59151 (local address 127.17.110.2:33801)
0807 18:17:56.375940 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:56.375947 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:56.376337 (+ 390us) server_negotiation.cc:407] Connection header received
0807 18:17:56.376628 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:56.376633 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:56.376704 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:56.376809 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:56.377366 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.378075 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.379997 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.380837 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.381025 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:56.381101 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:56.381396 (+ 295us) server_negotiation.cc:299] Negotiation successful
0807 18:17:56.381530 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 18:17:56.381971 6086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:56.375299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:33801 (local address 127.17.110.3:59151)
0807 18:17:56.376203 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:56.376219 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:56.376467 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:56.376827 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:56.376831 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:56.376857 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:56.377198 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:56.377207 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.378253 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.378257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:56.379820 (+ 1563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:56.379833 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.381056 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.381062 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:56.381224 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:56.381249 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 18:17:56.381364 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 18:17:56.381507 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":795,"mutex_wait_us":24,"spinlock_wait_cycles":35200,"thread_start_us":546,"threads_started":1}
I20260807 18:17:56.383111 5598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c543aee1eeed4714a0f8362effee8de7" candidate_uuid: "c8cf58c0a4ff4d22933a471b69be07e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeb211ad2c004cd0a3a40e510f67ac98" is_pre_election: true
I20260807 18:17:56.383111 5791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c543aee1eeed4714a0f8362effee8de7" candidate_uuid: "c8cf58c0a4ff4d22933a471b69be07e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9819a0a0990421994d36137ef0bf904" is_pre_election: true
I20260807 18:17:56.383739 5791 raft_consensus.cc:2454] T c543aee1eeed4714a0f8362effee8de7 P f9819a0a0990421994d36137ef0bf904 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8cf58c0a4ff4d22933a471b69be07e9 in term 0.
I20260807 18:17:56.383739 5598 raft_consensus.cc:2454] T c543aee1eeed4714a0f8362effee8de7 P aeb211ad2c004cd0a3a40e510f67ac98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8cf58c0a4ff4d22933a471b69be07e9 in term 0.
I20260807 18:17:56.384588 5894 leader_election.cc:302] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: aeb211ad2c004cd0a3a40e510f67ac98, c8cf58c0a4ff4d22933a471b69be07e9, f9819a0a0990421994d36137ef0bf904; no voters:
I20260807 18:17:56.385035 6085 raft_consensus.cc:2781] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:17:56.385217 6085 raft_consensus.cc:482] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:17:56.385349 6085 raft_consensus.cc:3037] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:56.387840 6085 raft_consensus.cc:504] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeb211ad2c004cd0a3a40e510f67ac98" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 44935 } } peers { permanent_uuid: "c8cf58c0a4ff4d22933a471b69be07e9" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 38927 } } peers { permanent_uuid: "f9819a0a0990421994d36137ef0bf904" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 33801 } }
I20260807 18:17:56.388659 6085 leader_election.cc:288] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9 [CANDIDATE]: Term 1 election: Requested vote from peers aeb211ad2c004cd0a3a40e510f67ac98 (127.17.110.1:44935), f9819a0a0990421994d36137ef0bf904 (127.17.110.2:33801)
I20260807 18:17:56.389197 5598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c543aee1eeed4714a0f8362effee8de7" candidate_uuid: "c8cf58c0a4ff4d22933a471b69be07e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeb211ad2c004cd0a3a40e510f67ac98"
I20260807 18:17:56.389263 5791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c543aee1eeed4714a0f8362effee8de7" candidate_uuid: "c8cf58c0a4ff4d22933a471b69be07e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9819a0a0990421994d36137ef0bf904"
I20260807 18:17:56.389459 5598 raft_consensus.cc:3037] T c543aee1eeed4714a0f8362effee8de7 P aeb211ad2c004cd0a3a40e510f67ac98 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:56.389519 5791 raft_consensus.cc:3037] T c543aee1eeed4714a0f8362effee8de7 P f9819a0a0990421994d36137ef0bf904 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:56.392208 5598 raft_consensus.cc:2454] T c543aee1eeed4714a0f8362effee8de7 P aeb211ad2c004cd0a3a40e510f67ac98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8cf58c0a4ff4d22933a471b69be07e9 in term 1.
I20260807 18:17:56.392339 5791 raft_consensus.cc:2454] T c543aee1eeed4714a0f8362effee8de7 P f9819a0a0990421994d36137ef0bf904 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8cf58c0a4ff4d22933a471b69be07e9 in term 1.
I20260807 18:17:56.392832 5898 leader_election.cc:302] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aeb211ad2c004cd0a3a40e510f67ac98, c8cf58c0a4ff4d22933a471b69be07e9; no voters:
I20260807 18:17:56.393235 6085 raft_consensus.cc:2781] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:17:56.393829 6085 raft_consensus.cc:686] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9 [term 1 LEADER]: Becoming Leader. State: Replica: c8cf58c0a4ff4d22933a471b69be07e9, State: Running, Role: LEADER
I20260807 18:17:56.394436 6085 consensus_queue.cc:237] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeb211ad2c004cd0a3a40e510f67ac98" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 44935 } } peers { permanent_uuid: "c8cf58c0a4ff4d22933a471b69be07e9" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 38927 } } peers { permanent_uuid: "f9819a0a0990421994d36137ef0bf904" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 33801 } }
I20260807 18:17:56.399834 5383 catalog_manager.cc:5495] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8cf58c0a4ff4d22933a471b69be07e9 (127.17.110.3). New cstate: current_term: 1 leader_uuid: "c8cf58c0a4ff4d22933a471b69be07e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeb211ad2c004cd0a3a40e510f67ac98" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 44935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8cf58c0a4ff4d22933a471b69be07e9" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 38927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9819a0a0990421994d36137ef0bf904" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 33801 } health_report { overall_health: UNKNOWN } } }
I20260807 18:17:56.463593 6070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:56.458017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44984 (local address 127.17.110.3:38927)
0807 18:17:56.458209 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:56.458214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:56.458232 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:17:56.458358 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:56.458362 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:56.458455 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:56.458600 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:17:56.459156 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.459908 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.461401 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.461814 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.461912 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:56.462444 (+ 532us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:17:56.463043 (+ 599us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:17:56.463163 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:56.463303 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 18:17:56.463382 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:17:56.473456 5791 raft_consensus.cc:1261] T c543aee1eeed4714a0f8362effee8de7 P f9819a0a0990421994d36137ef0bf904 [term 1 FOLLOWER]: Refusing update from remote peer c8cf58c0a4ff4d22933a471b69be07e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:17:56.473661 5598 raft_consensus.cc:1261] T c543aee1eeed4714a0f8362effee8de7 P aeb211ad2c004cd0a3a40e510f67ac98 [term 1 FOLLOWER]: Refusing update from remote peer c8cf58c0a4ff4d22933a471b69be07e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:17:56.475158 6085 consensus_queue.cc:1035] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aeb211ad2c004cd0a3a40e510f67ac98" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 44935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:17:56.475596 6087 consensus_queue.cc:1035] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9819a0a0990421994d36137ef0bf904" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 33801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:17:56.494856 6093 mvcc.cc:205] Tried to move back new op lower bound from 7315974866822205440 to 7315974866516123648. Current Snapshot: MvccSnapshot[applied={T|T < 7315974866822205440}]
I20260807 18:17:56.498885 6095 mvcc.cc:205] Tried to move back new op lower bound from 7315974866822205440 to 7315974866516123648. Current Snapshot: MvccSnapshot[applied={T|T < 7315974866822205440}]
I20260807 18:17:56.514927 5384 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:17:56.515304 5384 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:17:56.519922 5384 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906: Sending DeleteTablet for 3 replicas of tablet c543aee1eeed4714a0f8362effee8de7
I20260807 18:17:56.521718 5766 tablet_service.cc:1514] Processing DeleteTablet for tablet c543aee1eeed4714a0f8362effee8de7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:17:56 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:41862
I20260807 18:17:56.521715 5566 tablet_service.cc:1514] Processing DeleteTablet for tablet c543aee1eeed4714a0f8362effee8de7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:17:56 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:50670
I20260807 18:17:56.521965 5960 tablet_service.cc:1514] Processing DeleteTablet for tablet c543aee1eeed4714a0f8362effee8de7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:17:56 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:44980
I20260807 18:17:56.522805 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 5475
I20260807 18:17:56.535302 6107 tablet_replica.cc:330] T c543aee1eeed4714a0f8362effee8de7 P f9819a0a0990421994d36137ef0bf904: stopping tablet replica
I20260807 18:17:56.535832 6107 raft_consensus.cc:2229] T c543aee1eeed4714a0f8362effee8de7 P f9819a0a0990421994d36137ef0bf904 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:17:56.536487 6107 raft_consensus.cc:2258] T c543aee1eeed4714a0f8362effee8de7 P f9819a0a0990421994d36137ef0bf904 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:17:56.537771 6106 tablet_replica.cc:330] T c543aee1eeed4714a0f8362effee8de7 P aeb211ad2c004cd0a3a40e510f67ac98: stopping tablet replica
I20260807 18:17:56.538335 6106 raft_consensus.cc:2229] T c543aee1eeed4714a0f8362effee8de7 P aeb211ad2c004cd0a3a40e510f67ac98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:17:56.538934 6106 raft_consensus.cc:2258] T c543aee1eeed4714a0f8362effee8de7 P aeb211ad2c004cd0a3a40e510f67ac98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:17:56.539780 6109 tablet_replica.cc:330] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9: stopping tablet replica
I20260807 18:17:56.540251 6109 raft_consensus.cc:2229] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:17:56.540951 6109 raft_consensus.cc:2258] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:17:56.541648 6107 ts_tablet_manager.cc:1905] T c543aee1eeed4714a0f8362effee8de7 P f9819a0a0990421994d36137ef0bf904: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:17:56.542851 6106 ts_tablet_manager.cc:1905] T c543aee1eeed4714a0f8362effee8de7 P aeb211ad2c004cd0a3a40e510f67ac98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:17:56.543306 6109 ts_tablet_manager.cc:1905] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 18:17:56 dist-test-slave-39l8 krb5kdc[5321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.1@KRBTEST.COM
I20260807 18:17:56.551879 6109 ts_tablet_manager.cc:1918] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:17:56.552140 6109 log.cc:1196] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/wal/wals/c543aee1eeed4714a0f8362effee8de7
I20260807 18:17:56.552800 6109 ts_tablet_manager.cc:1939] T c543aee1eeed4714a0f8362effee8de7 P c8cf58c0a4ff4d22933a471b69be07e9: Deleting consensus metadata
I20260807 18:17:56.555035 5364 catalog_manager.cc:4841] TS c8cf58c0a4ff4d22933a471b69be07e9 (127.17.110.3:38927): tablet c543aee1eeed4714a0f8362effee8de7 (table test-table [id=d5717beeb0824c60b424819696be133c]) successfully deleted
I20260807 18:17:56.557381 6066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:56.524758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50686 (local address 127.17.110.1:44935)
0807 18:17:56.524983 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:56.524990 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:56.531467 (+ 6477us) server_negotiation.cc:407] Connection header received
0807 18:17:56.534119 (+ 2652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:56.534126 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:56.534425 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:56.534759 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:56.537588 (+ 2829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.538936 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.541125 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.541805 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.541923 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:56.548227 (+ 6304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:56.548251 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:56.548273 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:56.548337 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:56.553309 (+ 4972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:56.554491 (+ 1182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:56.554502 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:56.554505 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:56.554766 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:56.555351 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:56.555356 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:56.555359 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:56.555809 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:56.556058 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:56.556637 (+ 579us) server_negotiation.cc:299] Negotiation successful
0807 18:17:56.556885 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 18:17:56.558300 5653 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:56.558842 6107 ts_tablet_manager.cc:1918] T c543aee1eeed4714a0f8362effee8de7 P f9819a0a0990421994d36137ef0bf904: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:17:56.559049 6107 log.cc:1196] T c543aee1eeed4714a0f8362effee8de7 P f9819a0a0990421994d36137ef0bf904: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/wal/wals/c543aee1eeed4714a0f8362effee8de7
I20260807 18:17:56.559832 6107 ts_tablet_manager.cc:1939] T c543aee1eeed4714a0f8362effee8de7 P f9819a0a0990421994d36137ef0bf904: Deleting consensus metadata
I20260807 18:17:56.562799 5362 catalog_manager.cc:4841] TS f9819a0a0990421994d36137ef0bf904 (127.17.110.2:33801): tablet c543aee1eeed4714a0f8362effee8de7 (table test-table [id=d5717beeb0824c60b424819696be133c]) successfully deleted
I20260807 18:17:56.654620 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 5475
I20260807 18:17:56.689306 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 5671
W20260807 18:17:56.689253 5364 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.110.1:44935 (error 108)
I20260807 18:17:56.692873 6114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:56.692024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:44935 (local address 127.0.0.1:50692)
0807 18:17:56.692533 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:56.692668 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:44935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":270,"threads_started":1}
W20260807 18:17:56.693358 5364 catalog_manager.cc:4584] TS aeb211ad2c004cd0a3a40e510f67ac98 (127.17.110.1:44935): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c543aee1eeed4714a0f8362effee8de7: Network error: Client connection negotiation failed: client connection to 127.17.110.1:44935: connect: Connection refused (error 111)
Aug 07 18:17:56 dist-test-slave-39l8 krb5kdc[5321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.2@KRBTEST.COM
I20260807 18:17:56.715313 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:56.690265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41878 (local address 127.17.110.2:33801)
0807 18:17:56.690463 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:56.690470 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:56.690793 (+ 323us) server_negotiation.cc:407] Connection header received
0807 18:17:56.691106 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:56.691116 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:56.691236 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:56.691350 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:56.693438 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.694597 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.696424 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.697089 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.697221 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:56.707177 (+ 9956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:56.707205 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:56.707229 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:56.707299 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:56.711588 (+ 4289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:56.712627 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:56.712639 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:56.712643 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:56.712838 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:56.713346 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:56.713351 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:56.713355 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:56.713937 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:56.714274 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:56.714631 (+ 357us) server_negotiation.cc:299] Negotiation successful
0807 18:17:56.714940 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 18:17:56.716233 5853 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:56.747572 6114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:56.746974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:44935 (local address 127.0.0.1:50694)
0807 18:17:56.747245 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:56.747376 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:44935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 18:17:56.748967 6114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:56.748459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:44935 (local address 127.0.0.1:50700)
0807 18:17:56.748671 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:56.748776 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:44935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 18:17:56.809834 6114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:56.809168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:44935 (local address 127.0.0.1:50716)
0807 18:17:56.809491 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:56.809615 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:44935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260807 18:17:56.811339 6114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:56.810844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:44935 (local address 127.0.0.1:50726)
0807 18:17:56.811035 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:56.811134 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:44935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 18:17:56.814427 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 5671
I20260807 18:17:56.842089 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 5872
Aug 07 18:17:56 dist-test-slave-39l8 krb5kdc[5321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.3@KRBTEST.COM
I20260807 18:17:56.862748 6070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:56.843071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44986 (local address 127.17.110.3:38927)
0807 18:17:56.843363 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:56.843370 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:56.843395 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:17:56.843470 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:56.843475 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:56.843594 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:56.843714 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:56.845259 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.846558 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.848328 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.849061 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.849238 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:56.854418 (+ 5180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:56.854440 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:56.854465 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:56.854561 (+ 96us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:56.858966 (+ 4405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:56.859880 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:56.859895 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:56.859900 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:56.860112 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:56.860522 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:56.860527 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:56.860530 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:56.861098 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:56.861455 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:56.861982 (+ 527us) server_negotiation.cc:299] Negotiation successful
0807 18:17:56.862266 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260807 18:17:56.863831 6041 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:56.892429 6114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:56.891715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:44935 (local address 127.0.0.1:50736)
0807 18:17:56.892059 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:56.892248 (+ 189us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:44935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188}
I20260807 18:17:56.893639 6114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:56.893193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:44935 (local address 127.0.0.1:50742)
0807 18:17:56.893413 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:56.893488 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:44935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260807 18:17:56.990267 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 5872
I20260807 18:17:57.017372 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 5338
I20260807 18:17:57.018915 5433 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:57.133988 6114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:57.133075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:44935 (local address 127.0.0.1:50748)
0807 18:17:57.133481 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:57.133747 (+ 266us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:44935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199}
I20260807 18:17:57.134438 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 5338
I20260807 18:17:57.168619 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:42349
--webserver_interface=127.17.110.62
--webserver_port=38579
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:39359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:42349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:17:57.540359 6118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:57.540707 6118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:57.540876 6118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:57.548565 6118 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:17:57.548704 6118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:57.548795 6118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:57.548892 6118 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:17:57.548969 6118 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:17:57.559496 6118 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:39359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:42349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:42349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=38579
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:17:57.560962 6118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:57.562408 6118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:57.591718 6125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:57.591670 6128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:57.591657 6126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:57.593008 6118 server_base.cc:1034] running on GCE node
I20260807 18:17:57.594012 6118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:57.596009 6118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:57.597295 6118 hybrid_clock.cc:648] HybridClock initialized: now 1786126677597252 us; error 65 us; skew 500 ppm
I20260807 18:17:57.597819 6118 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:57 dist-test-slave-39l8 krb5kdc[5321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126677, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:57.602315 6118 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:17:57.604506 6118 webserver.cc:460] Webserver started at http://127.17.110.62:38579/ using document root <none> and password file <none>
I20260807 18:17:57.605432 6118 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:57.605633 6118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:57.612854 6118 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 18:17:57.616437 6139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:57.618296 6118 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:17:57.618556 6118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "c9a45c2d73e04aeca5f641cd1d765906"
format_stamp: "Formatted at 2026-08-07 18:17:54 on dist-test-slave-39l8"
server_key: "db76b691220b44585db6aef8f9bb8743"
server_key_iv: "573da03a6a251cfc5a44111b20a339a6"
server_key_version: "encryptionkey@0"
I20260807 18:17:57.619871 6118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:57.633692 6118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:57.639357 6118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:57.639743 6118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:57.666347 6118 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:42349
I20260807 18:17:57.666368 6216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:42349 every 8 connection(s)
I20260807 18:17:57.668519 6118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:17:57.672366 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 6118
I20260807 18:17:57.673322 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.1:44935
--local_ip_for_outbound_sockets=127.17.110.1
--tserver_master_addrs=127.17.110.62:42349
--webserver_port=44317
--webserver_interface=127.17.110.1
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:39359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:17:57.677246 6217 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:17:57.681118 6217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906: Bootstrap starting.
I20260807 18:17:57.708690 6217 log.cc:824] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:57.722406 6217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:17:57.723081 6217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906: Bootstrap complete.
I20260807 18:17:57.733510 6217 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9a45c2d73e04aeca5f641cd1d765906" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 42349 } }
I20260807 18:17:57.734488 6217 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9a45c2d73e04aeca5f641cd1d765906, State: Initialized, Role: FOLLOWER
I20260807 18:17:57.735329 6217 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9a45c2d73e04aeca5f641cd1d765906" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 42349 } }
I20260807 18:17:57.735598 6217 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:17:57.735744 6217 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:17:57.735944 6217 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:17:57.740336 6217 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9a45c2d73e04aeca5f641cd1d765906" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 42349 } }
I20260807 18:17:57.740990 6217 leader_election.cc:302] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9a45c2d73e04aeca5f641cd1d765906; no voters:
I20260807 18:17:57.741693 6217 leader_election.cc:288] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 18:17:57.741946 6223 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 18:17:57.743291 6223 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [term 2 LEADER]: Becoming Leader. State: Replica: c9a45c2d73e04aeca5f641cd1d765906, State: Running, Role: LEADER
I20260807 18:17:57.743937 6223 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9a45c2d73e04aeca5f641cd1d765906" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 42349 } }
I20260807 18:17:57.744208 6217 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:17:57.746762 6224 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c9a45c2d73e04aeca5f641cd1d765906" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9a45c2d73e04aeca5f641cd1d765906" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 42349 } } }
I20260807 18:17:57.747192 6224 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [sys.catalog]: This master's current role is: LEADER
I20260807 18:17:57.752140 6226 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9a45c2d73e04aeca5f641cd1d765906. Latest consensus state: current_term: 2 leader_uuid: "c9a45c2d73e04aeca5f641cd1d765906" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9a45c2d73e04aeca5f641cd1d765906" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 42349 } } }
I20260807 18:17:57.752640 6226 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [sys.catalog]: This master's current role is: LEADER
I20260807 18:17:57.753688 6232 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:17:57.764128 6232 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:17:57.765841 6232 catalog_manager.cc:1259] Loaded cluster ID: 2d92dac564b8453880b170c5b3a209fa
I20260807 18:17:57.765967 6232 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:17:57.769721 6232 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 18:17:57.789314 6232 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 18:17:57.799374 6232 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:17:57.801378 6232 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906: Loaded TSK: 0
I20260807 18:17:57.803359 6232 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 18:17:58.095999 6219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:58.096345 6219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:58.096506 6219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:58.104753 6219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:58.104899 6219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:58.105077 6219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.1
I20260807 18:17:58.116575 6219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:39359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.1:44935
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.110.1
--webserver_port=44317
--tserver_master_addrs=127.17.110.62:42349
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.1
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:17:58.118145 6219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:58.119779 6219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:58.137965 6260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:58.137984 6257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:58.138954 6219 server_base.cc:1034] running on GCE node
W20260807 18:17:58.143075 6256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:58.144289 6219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:58.145601 6219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:58.146860 6219 hybrid_clock.cc:648] HybridClock initialized: now 1786126678146814 us; error 53 us; skew 500 ppm
I20260807 18:17:58.147246 6219 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:58 dist-test-slave-39l8 krb5kdc[5321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126678, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:58.151268 6219 init.cc:377] Logged in from keytab as kudu/127.17.110.1@KRBTEST.COM (short username kudu)
I20260807 18:17:58.153268 6219 webserver.cc:460] Webserver started at http://127.17.110.1:44317/ using document root <none> and password file <none>
I20260807 18:17:58.154135 6219 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:58.154323 6219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:58.160828 6219 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:17:58.164950 6270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:58.166654 6219 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:17:58.166927 6219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/wal
uuid: "aeb211ad2c004cd0a3a40e510f67ac98"
format_stamp: "Formatted at 2026-08-07 18:17:54 on dist-test-slave-39l8"
server_key: "a931ebb27a09180b4bc7c57e28a17834"
server_key_iv: "f602632313b312703a42ac4e7cda1a48"
server_key_version: "encryptionkey@0"
I20260807 18:17:58.167995 6219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:58.189836 6219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:58.195214 6219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:58.195523 6219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:58.196692 6219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:17:58.199047 6283 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260807 18:17:58.205592 6283 ts_tablet_manager.cc:1852] T c543aee1eeed4714a0f8362effee8de7 P aeb211ad2c004cd0a3a40e510f67ac98: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260807 18:17:58.206718 6283 ts_tablet_manager.cc:1905] T c543aee1eeed4714a0f8362effee8de7 P aeb211ad2c004cd0a3a40e510f67ac98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:17:58.215127 6283 ts_tablet_manager.cc:1918] T c543aee1eeed4714a0f8362effee8de7 P aeb211ad2c004cd0a3a40e510f67ac98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:17:58.215433 6283 log.cc:1196] T c543aee1eeed4714a0f8362effee8de7 P aeb211ad2c004cd0a3a40e510f67ac98: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/wal/wals/c543aee1eeed4714a0f8362effee8de7
I20260807 18:17:58.215982 6283 ts_tablet_manager.cc:1939] T c543aee1eeed4714a0f8362effee8de7 P aeb211ad2c004cd0a3a40e510f67ac98: Deleting consensus metadata
I20260807 18:17:58.216459 6219 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260807 18:17:58.216588 6219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.018s user 0.001s sys 0.000s
I20260807 18:17:58.216707 6219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:17:58.216771 6219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:58.255064 6219 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.1:44935
I20260807 18:17:58.255219 6435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.1:44935 every 8 connection(s)
I20260807 18:17:58.256589 6219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
I20260807 18:17:58.262148 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 6219
I20260807 18:17:58.262884 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.2:33801
--local_ip_for_outbound_sockets=127.17.110.2
--tserver_master_addrs=127.17.110.62:42349
--webserver_port=36373
--webserver_interface=127.17.110.2
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:39359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:17:58 dist-test-slave-39l8 krb5kdc[5321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126678, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:17:58.285981 6440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:58.259785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:40697 (local address 127.17.110.62:42349)
0807 18:17:58.260402 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:58.260414 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:58.261201 (+ 787us) server_negotiation.cc:407] Connection header received
0807 18:17:58.263788 (+ 2587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:58.263812 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:58.264353 (+ 541us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:58.264818 (+ 465us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:58.266439 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:58.268559 (+ 2120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:58.270600 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:58.271865 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:58.272141 (+ 276us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:58.277084 (+ 4943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:58.277094 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:58.277112 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:58.277163 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:58.280595 (+ 3432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:58.281651 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:58.281659 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:58.281666 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:58.281847 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:58.282369 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:58.282375 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:58.282380 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:58.283030 (+ 650us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:58.283253 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:58.284143 (+ 890us) server_negotiation.cc:299] Negotiation successful
0807 18:17:58.284602 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":266,"threads_started":1}
I20260807 18:17:58.286019 6441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:58.260004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.1:40697)
0807 18:17:58.260916 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:58.260978 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:58.263093 (+ 2115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:58.264953 (+ 1860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:58.264961 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:58.265556 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:58.266167 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:58.266189 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:58.268837 (+ 2648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:58.268844 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:58.270385 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:58.270406 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:58.272300 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:58.272309 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:58.272598 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:58.273446 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:17:58.273508 (+ 62us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:58.276715 (+ 3207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:58.280837 (+ 4122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:58.280848 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:58.280874 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:58.281413 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:58.282015 (+ 602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:58.282022 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:58.282030 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:58.282235 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:58.283235 (+ 1000us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:58.283246 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:58.283612 (+ 366us) client_negotiation.cc:769] Sending connection context
0807 18:17:58.283919 (+ 307us) client_negotiation.cc:240] Negotiation successful
0807 18:17:58.284599 (+ 680us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":376,"threads_started":1}
I20260807 18:17:58.289736 6436 heartbeater.cc:347] Connected to a master server at 127.17.110.62:42349
I20260807 18:17:58.290104 6436 heartbeater.cc:464] Registering TS with master...
I20260807 18:17:58.291031 6436 heartbeater.cc:511] Master 127.17.110.62:42349 requested a full tablet report, sending...
I20260807 18:17:58.293998 6162 ts_manager.cc:194] Registered new tserver with Master: aeb211ad2c004cd0a3a40e510f67ac98 (127.17.110.1:44935)
I20260807 18:17:58.296378 6162 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.1@KRBTEST.COM'} at 127.17.110.1:40697
W20260807 18:17:58.644217 6443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:58.644651 6443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:58.644805 6443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:58.652590 6443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:58.652782 6443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:58.652974 6443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.2
I20260807 18:17:58.663473 6443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:39359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.2:33801
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.110.2
--webserver_port=36373
--tserver_master_addrs=127.17.110.62:42349
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.2
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:17:58.664942 6443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:58.666460 6443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:58.681252 6451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:58.681905 6450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:58.682816 6455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:58.683207 6443 server_base.cc:1034] running on GCE node
I20260807 18:17:58.683988 6443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:58.685357 6443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:58.686669 6443 hybrid_clock.cc:648] HybridClock initialized: now 1786126678686590 us; error 58 us; skew 500 ppm
I20260807 18:17:58.687150 6443 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:58 dist-test-slave-39l8 krb5kdc[5321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126678, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:58.691164 6443 init.cc:377] Logged in from keytab as kudu/127.17.110.2@KRBTEST.COM (short username kudu)
I20260807 18:17:58.692917 6443 webserver.cc:460] Webserver started at http://127.17.110.2:36373/ using document root <none> and password file <none>
I20260807 18:17:58.693722 6443 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:58.693861 6443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:58.700795 6443 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 18:17:58.704514 6463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:58.706426 6443 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:17:58.706619 6443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/wal
uuid: "f9819a0a0990421994d36137ef0bf904"
format_stamp: "Formatted at 2026-08-07 18:17:55 on dist-test-slave-39l8"
server_key: "8840004cfcbfe9919dab3cf453acf8e4"
server_key_iv: "9e5d3a254e8862dfe80a99555d35578d"
server_key_version: "encryptionkey@0"
I20260807 18:17:58.707855 6443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:58.736475 6443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:58.741792 6443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:58.742086 6443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:58.743283 6443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:17:58.744930 6443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:17:58.745042 6443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:58.745163 6443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:17:58.745263 6443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:58.786201 6443 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.2:33801
I20260807 18:17:58.786324 6619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.2:33801 every 8 connection(s)
I20260807 18:17:58.787993 6443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
I20260807 18:17:58.797775 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 6443
I20260807 18:17:58.798616 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.3:38927
--local_ip_for_outbound_sockets=127.17.110.3
--tserver_master_addrs=127.17.110.62:42349
--webserver_port=44603
--webserver_interface=127.17.110.3
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:39359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:17:58 dist-test-slave-39l8 krb5kdc[5321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126678, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:17:58.813642 6622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:58.791787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:45685 (local address 127.17.110.62:42349)
0807 18:17:58.792508 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:58.792513 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:58.793322 (+ 809us) server_negotiation.cc:407] Connection header received
0807 18:17:58.795441 (+ 2119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:58.795446 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:58.795551 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:58.795671 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:58.797429 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:58.798604 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:58.800684 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:58.801456 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:58.801623 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:58.806360 (+ 4737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:58.806366 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:58.806370 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:58.806413 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:58.809624 (+ 3211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:58.810592 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:58.810602 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:58.810609 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:58.810836 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:58.811316 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:58.811324 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:58.811330 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:58.811766 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:58.811927 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:58.812731 (+ 804us) server_negotiation.cc:299] Negotiation successful
0807 18:17:58.813015 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":405,"threads_started":1}
I20260807 18:17:58.814599 6623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:58.792190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.2:45685)
0807 18:17:58.793093 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:58.793145 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:58.795108 (+ 1963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:58.795907 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:58.795916 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:58.796525 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:58.797180 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:58.797198 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:58.798886 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:58.798895 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:58.800503 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:58.800517 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:58.801730 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:58.801736 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:58.801965 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:58.802985 (+ 1020us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:17:58.803015 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:58.805995 (+ 2980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:58.809864 (+ 3869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:58.809873 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:58.809890 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:58.810433 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:58.810974 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:58.810978 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:58.810980 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:58.811193 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:58.811919 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:58.811926 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:58.812325 (+ 399us) client_negotiation.cc:769] Sending connection context
0807 18:17:58.812586 (+ 261us) client_negotiation.cc:240] Negotiation successful
0807 18:17:58.813104 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":384,"threads_started":1}
I20260807 18:17:58.816425 6620 heartbeater.cc:347] Connected to a master server at 127.17.110.62:42349
I20260807 18:17:58.816795 6620 heartbeater.cc:464] Registering TS with master...
I20260807 18:17:58.817626 6620 heartbeater.cc:511] Master 127.17.110.62:42349 requested a full tablet report, sending...
I20260807 18:17:58.819381 6162 ts_manager.cc:194] Registered new tserver with Master: f9819a0a0990421994d36137ef0bf904 (127.17.110.2:33801)
I20260807 18:17:58.820261 6162 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.2@KRBTEST.COM'} at 127.17.110.2:45685
W20260807 18:17:59.166538 6624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:59.167016 6624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:59.167178 6624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:59.175102 6624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:59.175241 6624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:59.175419 6624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.3
I20260807 18:17:59.185704 6624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:39359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.3:38927
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.110.3
--webserver_port=44603
--tserver_master_addrs=127.17.110.62:42349
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.3
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:17:59.187306 6624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:59.188897 6624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:59.204200 6636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:59.204672 6632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:59.204738 6639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:59.205423 6624 server_base.cc:1034] running on GCE node
I20260807 18:17:59.206137 6624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:59.207367 6624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:59.208686 6624 hybrid_clock.cc:648] HybridClock initialized: now 1786126679208605 us; error 75 us; skew 500 ppm
I20260807 18:17:59.209164 6624 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:59 dist-test-slave-39l8 krb5kdc[5321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126679, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:59.213138 6624 init.cc:377] Logged in from keytab as kudu/127.17.110.3@KRBTEST.COM (short username kudu)
I20260807 18:17:59.214928 6624 webserver.cc:460] Webserver started at http://127.17.110.3:44603/ using document root <none> and password file <none>
I20260807 18:17:59.215715 6624 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:59.215880 6624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:59.222187 6624 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.002s
I20260807 18:17:59.225986 6647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:59.227939 6624 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260807 18:17:59.228140 6624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/wal
uuid: "c8cf58c0a4ff4d22933a471b69be07e9"
format_stamp: "Formatted at 2026-08-07 18:17:55 on dist-test-slave-39l8"
server_key: "4a892bbf4046e5805f4eaa8133c2f917"
server_key_iv: "e389a16ce06e2a341ee4370d96a3cd3e"
server_key_version: "encryptionkey@0"
I20260807 18:17:59.229295 6624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:59.259531 6624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:59.264684 6624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:59.264951 6624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:59.266008 6624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:17:59.267750 6624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:17:59.267853 6624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:59.267931 6624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:17:59.267987 6624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:59.300467 6436 heartbeater.cc:503] Master 127.17.110.62:42349 was elected leader, sending a full tablet report...
I20260807 18:17:59.308111 6624 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.3:38927
I20260807 18:17:59.308152 6815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.3:38927 every 8 connection(s)
I20260807 18:17:59.309799 6624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
I20260807 18:17:59.313583 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 6624
Aug 07 18:17:59 dist-test-slave-39l8 krb5kdc[5321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126679, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:17:59.335500 6820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:59.314671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47418 (local address 127.17.110.62:42349)
0807 18:17:59.315762 (+ 1091us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:59.315769 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:59.316441 (+ 672us) server_negotiation.cc:407] Connection header received
0807 18:17:59.316691 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:59.316695 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:59.316768 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:59.316880 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:59.319501 (+ 2621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.320357 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.322259 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.322769 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.322894 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:59.327286 (+ 4392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:59.327292 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:59.327296 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:59.327334 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:59.331300 (+ 3966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:59.332670 (+ 1370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:59.332679 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:59.332683 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:59.332907 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:59.333452 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:59.333458 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:59.333460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:59.333845 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:59.334212 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:59.334648 (+ 436us) server_negotiation.cc:299] Negotiation successful
0807 18:17:59.334958 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":961,"spinlock_wait_cycles":6912,"thread_start_us":746,"threads_started":1}
I20260807 18:17:59.338292 6622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:59.313295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:55331 (local address 127.17.110.62:42349)
0807 18:17:59.313575 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:59.313583 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:59.315054 (+ 1471us) server_negotiation.cc:407] Connection header received
0807 18:17:59.318696 (+ 3642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:59.318701 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:59.318864 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:59.318959 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:59.321075 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.321778 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.324236 (+ 2458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.324696 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.324864 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:59.329969 (+ 5105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:59.329980 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:59.329987 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:59.330059 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:59.333796 (+ 3737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:59.334935 (+ 1139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:59.334939 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:59.334942 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:59.335063 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:59.335733 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:59.335738 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:59.335741 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:59.336113 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:59.336312 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:59.337652 (+ 1340us) server_negotiation.cc:299] Negotiation successful
0807 18:17:59.337931 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260807 18:17:59.339488 6819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:59.313840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.3:55331)
0807 18:17:59.314878 (+ 1038us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:59.314931 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:59.318282 (+ 3351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:59.319345 (+ 1063us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:59.319362 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:59.320011 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:59.320858 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:59.320889 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.321978 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.321985 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:59.324048 (+ 2063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:59.324067 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.325014 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.325025 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:59.325429 (+ 404us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:59.326323 (+ 894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:17:59.326365 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:59.329665 (+ 3300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:59.333971 (+ 4306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:59.333983 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:59.334004 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:59.334774 (+ 770us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:59.335202 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:59.335207 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:59.335211 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:59.335539 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:59.336341 (+ 802us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:59.336353 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:59.337014 (+ 661us) client_negotiation.cc:769] Sending connection context
0807 18:17:59.337439 (+ 425us) client_negotiation.cc:240] Negotiation successful
0807 18:17:59.337985 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"thread_start_us":426,"threads_started":1}
I20260807 18:17:59.341334 6816 heartbeater.cc:347] Connected to a master server at 127.17.110.62:42349
I20260807 18:17:59.341722 6816 heartbeater.cc:464] Registering TS with master...
I20260807 18:17:59.342603 6816 heartbeater.cc:511] Master 127.17.110.62:42349 requested a full tablet report, sending...
I20260807 18:17:59.344624 6162 ts_manager.cc:194] Registered new tserver with Master: c8cf58c0a4ff4d22933a471b69be07e9 (127.17.110.3:38927)
I20260807 18:17:59.345619 6162 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.3@KRBTEST.COM'} at 127.17.110.3:55331
I20260807 18:17:59.352408 17848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:17:59.360949 6622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:59.353705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47426 (local address 127.17.110.62:42349)
0807 18:17:59.353901 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:59.353906 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:59.354282 (+ 376us) server_negotiation.cc:407] Connection header received
0807 18:17:59.354575 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:59.354580 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:59.354799 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:59.354956 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:17:59.355582 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.356401 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.358344 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.358910 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.359029 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:59.359676 (+ 647us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:17:59.360288 (+ 612us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:17:59.360423 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:59.360612 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 18:17:59.360705 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 18:17:59.362193 6162 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:47426:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:17:59.371089 6162 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:17:59.413998 6840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:59.387431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41880 (local address 127.17.110.2:33801)
0807 18:17:59.400321 (+ 12890us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:59.400333 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:59.402315 (+ 1982us) server_negotiation.cc:407] Connection header received
0807 18:17:59.402442 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:59.402450 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:59.402780 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:59.403001 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:59.404581 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.406891 (+ 2310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.410618 (+ 3727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.412257 (+ 1639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.412414 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:59.412635 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:59.413445 (+ 810us) server_negotiation.cc:299] Negotiation successful
0807 18:17:59.413671 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12606,"thread_start_us":14146,"threads_started":1}
I20260807 18:17:59.414013 6830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:59.386443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:38927 (local address 127.0.0.1:44996)
0807 18:17:59.401044 (+ 14601us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:59.401123 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:59.401317 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:59.403390 (+ 2073us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:59.403401 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:59.403494 (+ 93us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:59.404263 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:59.404283 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.406797 (+ 2514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.406803 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:59.410479 (+ 3676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:59.410498 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.412785 (+ 2287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.412792 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:59.413113 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:59.413182 (+ 69us) client_negotiation.cc:769] Sending connection context
0807 18:17:59.413328 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 18:17:59.413561 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14262,"mutex_wait_us":70,"spinlock_wait_cycles":6144,"thread_start_us":15778,"threads_started":1}
I20260807 18:17:59.414875 6831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:59.386814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44996 (local address 127.17.110.3:38927)
0807 18:17:59.402387 (+ 15573us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:59.402396 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:59.402429 (+ 33us) server_negotiation.cc:407] Connection header received
0807 18:17:59.402527 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:59.402533 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:59.402815 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:59.403022 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:59.404613 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.406544 (+ 1931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.410763 (+ 4219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.412145 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.412289 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:59.412573 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:59.413974 (+ 1401us) server_negotiation.cc:299] Negotiation successful
0807 18:17:59.414468 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15381,"thread_start_us":15196,"threads_started":1}
I20260807 18:17:59.415949 6828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:59.386782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:33801 (local address 127.0.0.1:41880)
0807 18:17:59.401957 (+ 15175us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:59.401989 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:59.402146 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:59.403390 (+ 1244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:59.403402 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:59.403479 (+ 77us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:59.404241 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:59.404270 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.407171 (+ 2901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.407180 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:59.410360 (+ 3180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:59.410381 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.412854 (+ 2473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.412860 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:59.413202 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:59.413243 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 18:17:59.414333 (+ 1090us) client_negotiation.cc:240] Negotiation successful
0807 18:17:59.414501 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14987,"mutex_wait_us":95,"spinlock_wait_cycles":25088,"thread_start_us":15019,"threads_started":1}
I20260807 18:17:59.419093 6844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:59.403193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50774 (local address 127.17.110.1:44935)
0807 18:17:59.410766 (+ 7573us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:59.410775 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:59.410808 (+ 33us) server_negotiation.cc:407] Connection header received
0807 18:17:59.410917 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:59.410926 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:59.411253 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:59.411497 (+ 244us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:59.412347 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.413947 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.416431 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.417805 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.417995 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:59.418270 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:59.418552 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 18:17:59.418751 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7337,"thread_start_us":7181,"threads_started":1}
I20260807 18:17:59.419091 6842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:59.402842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:44935 (local address 127.0.0.1:50774)
0807 18:17:59.409705 (+ 6863us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:59.409730 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:59.409895 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:59.411519 (+ 1624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:59.411526 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:59.411560 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:59.412052 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:59.412069 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.414144 (+ 2075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.414150 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:59.416034 (+ 1884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:59.416057 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.418081 (+ 2024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.418089 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:59.418298 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:59.418340 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 18:17:59.418471 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 18:17:59.418656 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6673,"thread_start_us":7154,"threads_started":1}
I20260807 18:17:59.419392 6726 tablet_service.cc:1467] Processing CreateTablet for tablet 97cae5f163e64242b62587b99ef8c14b (DEFAULT_TABLE table=test-table [id=83f6e771d9494bd5b54c82188b582ebb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:17:59.420240 6540 tablet_service.cc:1467] Processing CreateTablet for tablet 97cae5f163e64242b62587b99ef8c14b (DEFAULT_TABLE table=test-table [id=83f6e771d9494bd5b54c82188b582ebb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:17:59.421164 6726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97cae5f163e64242b62587b99ef8c14b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:59.421711 6540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97cae5f163e64242b62587b99ef8c14b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:59.421698 6343 tablet_service.cc:1467] Processing CreateTablet for tablet 97cae5f163e64242b62587b99ef8c14b (DEFAULT_TABLE table=test-table [id=83f6e771d9494bd5b54c82188b582ebb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:17:59.422964 6343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97cae5f163e64242b62587b99ef8c14b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:59.433446 6845 tablet_bootstrap.cc:492] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98: Bootstrap starting.
I20260807 18:17:59.437592 6845 tablet_bootstrap.cc:654] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:59.439018 6845 log.cc:824] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:59.439378 6846 tablet_bootstrap.cc:492] T 97cae5f163e64242b62587b99ef8c14b P f9819a0a0990421994d36137ef0bf904: Bootstrap starting.
I20260807 18:17:59.443657 6848 tablet_bootstrap.cc:492] T 97cae5f163e64242b62587b99ef8c14b P c8cf58c0a4ff4d22933a471b69be07e9: Bootstrap starting.
I20260807 18:17:59.444293 6845 tablet_bootstrap.cc:492] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98: No bootstrap required, opened a new log
I20260807 18:17:59.444645 6845 ts_tablet_manager.cc:1397] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260807 18:17:59.444989 6846 tablet_bootstrap.cc:654] T 97cae5f163e64242b62587b99ef8c14b P f9819a0a0990421994d36137ef0bf904: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:59.446455 6846 log.cc:824] T 97cae5f163e64242b62587b99ef8c14b P f9819a0a0990421994d36137ef0bf904: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:59.447732 6848 tablet_bootstrap.cc:654] T 97cae5f163e64242b62587b99ef8c14b P c8cf58c0a4ff4d22933a471b69be07e9: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:59.449193 6848 log.cc:824] T 97cae5f163e64242b62587b99ef8c14b P c8cf58c0a4ff4d22933a471b69be07e9: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:59.450285 6846 tablet_bootstrap.cc:492] T 97cae5f163e64242b62587b99ef8c14b P f9819a0a0990421994d36137ef0bf904: No bootstrap required, opened a new log
I20260807 18:17:59.450618 6846 ts_tablet_manager.cc:1397] T 97cae5f163e64242b62587b99ef8c14b P f9819a0a0990421994d36137ef0bf904: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260807 18:17:59.452809 6848 tablet_bootstrap.cc:492] T 97cae5f163e64242b62587b99ef8c14b P c8cf58c0a4ff4d22933a471b69be07e9: No bootstrap required, opened a new log
I20260807 18:17:59.453142 6848 ts_tablet_manager.cc:1397] T 97cae5f163e64242b62587b99ef8c14b P c8cf58c0a4ff4d22933a471b69be07e9: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 18:17:59.454952 6845 raft_consensus.cc:348] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8cf58c0a4ff4d22933a471b69be07e9" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 38927 } } peers { permanent_uuid: "f9819a0a0990421994d36137ef0bf904" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 33801 } } peers { permanent_uuid: "aeb211ad2c004cd0a3a40e510f67ac98" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 44935 } }
I20260807 18:17:59.455475 6845 raft_consensus.cc:374] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:59.455612 6845 raft_consensus.cc:729] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aeb211ad2c004cd0a3a40e510f67ac98, State: Initialized, Role: FOLLOWER
I20260807 18:17:59.456410 6845 consensus_queue.cc:260] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8cf58c0a4ff4d22933a471b69be07e9" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 38927 } } peers { permanent_uuid: "f9819a0a0990421994d36137ef0bf904" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 33801 } } peers { permanent_uuid: "aeb211ad2c004cd0a3a40e510f67ac98" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 44935 } }
I20260807 18:17:59.458415 6845 ts_tablet_manager.cc:1428] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260807 18:17:59.460184 6846 raft_consensus.cc:348] T 97cae5f163e64242b62587b99ef8c14b P f9819a0a0990421994d36137ef0bf904 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8cf58c0a4ff4d22933a471b69be07e9" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 38927 } } peers { permanent_uuid: "f9819a0a0990421994d36137ef0bf904" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 33801 } } peers { permanent_uuid: "aeb211ad2c004cd0a3a40e510f67ac98" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 44935 } }
I20260807 18:17:59.460626 6846 raft_consensus.cc:374] T 97cae5f163e64242b62587b99ef8c14b P f9819a0a0990421994d36137ef0bf904 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:59.460755 6846 raft_consensus.cc:729] T 97cae5f163e64242b62587b99ef8c14b P f9819a0a0990421994d36137ef0bf904 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9819a0a0990421994d36137ef0bf904, State: Initialized, Role: FOLLOWER
I20260807 18:17:59.461439 6846 consensus_queue.cc:260] T 97cae5f163e64242b62587b99ef8c14b P f9819a0a0990421994d36137ef0bf904 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8cf58c0a4ff4d22933a471b69be07e9" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 38927 } } peers { permanent_uuid: "f9819a0a0990421994d36137ef0bf904" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 33801 } } peers { permanent_uuid: "aeb211ad2c004cd0a3a40e510f67ac98" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 44935 } }
I20260807 18:17:59.462742 6620 heartbeater.cc:503] Master 127.17.110.62:42349 was elected leader, sending a full tablet report...
I20260807 18:17:59.463616 6846 ts_tablet_manager.cc:1428] T 97cae5f163e64242b62587b99ef8c14b P f9819a0a0990421994d36137ef0bf904: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260807 18:17:59.464253 6848 raft_consensus.cc:348] T 97cae5f163e64242b62587b99ef8c14b P c8cf58c0a4ff4d22933a471b69be07e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8cf58c0a4ff4d22933a471b69be07e9" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 38927 } } peers { permanent_uuid: "f9819a0a0990421994d36137ef0bf904" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 33801 } } peers { permanent_uuid: "aeb211ad2c004cd0a3a40e510f67ac98" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 44935 } }
I20260807 18:17:59.464746 6848 raft_consensus.cc:374] T 97cae5f163e64242b62587b99ef8c14b P c8cf58c0a4ff4d22933a471b69be07e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:59.464864 6848 raft_consensus.cc:729] T 97cae5f163e64242b62587b99ef8c14b P c8cf58c0a4ff4d22933a471b69be07e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8cf58c0a4ff4d22933a471b69be07e9, State: Initialized, Role: FOLLOWER
I20260807 18:17:59.465595 6848 consensus_queue.cc:260] T 97cae5f163e64242b62587b99ef8c14b P c8cf58c0a4ff4d22933a471b69be07e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8cf58c0a4ff4d22933a471b69be07e9" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 38927 } } peers { permanent_uuid: "f9819a0a0990421994d36137ef0bf904" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 33801 } } peers { permanent_uuid: "aeb211ad2c004cd0a3a40e510f67ac98" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 44935 } }
I20260807 18:17:59.466979 6816 heartbeater.cc:503] Master 127.17.110.62:42349 was elected leader, sending a full tablet report...
I20260807 18:17:59.467636 6848 ts_tablet_manager.cc:1428] T 97cae5f163e64242b62587b99ef8c14b P c8cf58c0a4ff4d22933a471b69be07e9: Time spent starting tablet: real 0.014s user 0.010s sys 0.006s
I20260807 18:17:59.471702 6852 raft_consensus.cc:482] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:17:59.472050 6852 raft_consensus.cc:504] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8cf58c0a4ff4d22933a471b69be07e9" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 38927 } } peers { permanent_uuid: "f9819a0a0990421994d36137ef0bf904" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 33801 } } peers { permanent_uuid: "aeb211ad2c004cd0a3a40e510f67ac98" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 44935 } }
I20260807 18:17:59.473793 6852 leader_election.cc:288] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8cf58c0a4ff4d22933a471b69be07e9 (127.17.110.3:38927), f9819a0a0990421994d36137ef0bf904 (127.17.110.2:33801)
I20260807 18:17:59.481822 6858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:59.474055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:38927 (local address 127.17.110.1:42601)
0807 18:17:59.475093 (+ 1038us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:59.475110 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:59.475301 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:59.475734 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:59.475740 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:59.475776 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:59.476333 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:59.476348 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.477685 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.477691 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:59.479504 (+ 1813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:59.479522 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.480960 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.480968 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:59.481180 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:59.481231 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 18:17:59.481334 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 18:17:59.481497 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":912,"thread_start_us":710,"threads_started":1}
I20260807 18:17:59.481825 6831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:59.474503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:42601 (local address 127.17.110.3:38927)
0807 18:17:59.474769 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:59.474778 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:59.475219 (+ 441us) server_negotiation.cc:407] Connection header received
0807 18:17:59.475460 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:59.475468 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:59.475563 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:59.475670 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:59.476516 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.477539 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.479697 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.480722 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.480867 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:59.480925 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:59.481388 (+ 463us) server_negotiation.cc:299] Negotiation successful
0807 18:17:59.481537 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 18:17:59.482306 6840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:59.474286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:35461 (local address 127.17.110.2:33801)
0807 18:17:59.474564 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:59.474574 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:59.476113 (+ 1539us) server_negotiation.cc:407] Connection header received
0807 18:17:59.476346 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:59.476351 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:59.476445 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:59.476539 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:59.477190 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.478101 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.480203 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.481160 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.481336 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:59.481406 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:59.481888 (+ 482us) server_negotiation.cc:299] Negotiation successful
0807 18:17:59.482042 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 18:17:59.482409 6856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:59.474055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:33801 (local address 127.17.110.1:35461)
0807 18:17:59.475998 (+ 1943us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:59.476013 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:59.476179 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:59.476563 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:59.476568 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:59.476601 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:59.477005 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:59.477020 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.478281 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.478286 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:59.480007 (+ 1721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:59.480023 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.481442 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.481449 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:59.481684 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:59.481720 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 18:17:59.481834 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 18:17:59.481988 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1806,"mutex_wait_us":47,"thread_start_us":1443,"threads_started":1}
I20260807 18:17:59.483385 6752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cae5f163e64242b62587b99ef8c14b" candidate_uuid: "aeb211ad2c004cd0a3a40e510f67ac98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8cf58c0a4ff4d22933a471b69be07e9" is_pre_election: true
I20260807 18:17:59.483450 6566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cae5f163e64242b62587b99ef8c14b" candidate_uuid: "aeb211ad2c004cd0a3a40e510f67ac98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9819a0a0990421994d36137ef0bf904" is_pre_election: true
I20260807 18:17:59.483919 6752 raft_consensus.cc:2454] T 97cae5f163e64242b62587b99ef8c14b P c8cf58c0a4ff4d22933a471b69be07e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aeb211ad2c004cd0a3a40e510f67ac98 in term 0.
I20260807 18:17:59.483945 6566 raft_consensus.cc:2454] T 97cae5f163e64242b62587b99ef8c14b P f9819a0a0990421994d36137ef0bf904 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aeb211ad2c004cd0a3a40e510f67ac98 in term 0.
I20260807 18:17:59.484679 6278 leader_election.cc:302] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aeb211ad2c004cd0a3a40e510f67ac98, c8cf58c0a4ff4d22933a471b69be07e9; no voters:
I20260807 18:17:59.485164 6852 raft_consensus.cc:2781] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:17:59.485325 6852 raft_consensus.cc:482] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:17:59.485437 6852 raft_consensus.cc:3037] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:59.487802 6852 raft_consensus.cc:504] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8cf58c0a4ff4d22933a471b69be07e9" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 38927 } } peers { permanent_uuid: "f9819a0a0990421994d36137ef0bf904" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 33801 } } peers { permanent_uuid: "aeb211ad2c004cd0a3a40e510f67ac98" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 44935 } }
I20260807 18:17:59.488510 6852 leader_election.cc:288] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98 [CANDIDATE]: Term 1 election: Requested vote from peers c8cf58c0a4ff4d22933a471b69be07e9 (127.17.110.3:38927), f9819a0a0990421994d36137ef0bf904 (127.17.110.2:33801)
I20260807 18:17:59.489095 6752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cae5f163e64242b62587b99ef8c14b" candidate_uuid: "aeb211ad2c004cd0a3a40e510f67ac98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8cf58c0a4ff4d22933a471b69be07e9"
I20260807 18:17:59.489164 6566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cae5f163e64242b62587b99ef8c14b" candidate_uuid: "aeb211ad2c004cd0a3a40e510f67ac98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9819a0a0990421994d36137ef0bf904"
I20260807 18:17:59.489360 6752 raft_consensus.cc:3037] T 97cae5f163e64242b62587b99ef8c14b P c8cf58c0a4ff4d22933a471b69be07e9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:59.489432 6566 raft_consensus.cc:3037] T 97cae5f163e64242b62587b99ef8c14b P f9819a0a0990421994d36137ef0bf904 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:59.492282 6752 raft_consensus.cc:2454] T 97cae5f163e64242b62587b99ef8c14b P c8cf58c0a4ff4d22933a471b69be07e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aeb211ad2c004cd0a3a40e510f67ac98 in term 1.
I20260807 18:17:59.492509 6566 raft_consensus.cc:2454] T 97cae5f163e64242b62587b99ef8c14b P f9819a0a0990421994d36137ef0bf904 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aeb211ad2c004cd0a3a40e510f67ac98 in term 1.
I20260807 18:17:59.493211 6278 leader_election.cc:302] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aeb211ad2c004cd0a3a40e510f67ac98, c8cf58c0a4ff4d22933a471b69be07e9; no voters:
I20260807 18:17:59.493757 6852 raft_consensus.cc:2781] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:17:59.494311 6852 raft_consensus.cc:686] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98 [term 1 LEADER]: Becoming Leader. State: Replica: aeb211ad2c004cd0a3a40e510f67ac98, State: Running, Role: LEADER
I20260807 18:17:59.494923 6852 consensus_queue.cc:237] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8cf58c0a4ff4d22933a471b69be07e9" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 38927 } } peers { permanent_uuid: "f9819a0a0990421994d36137ef0bf904" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 33801 } } peers { permanent_uuid: "aeb211ad2c004cd0a3a40e510f67ac98" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 44935 } }
I20260807 18:17:59.500118 6161 catalog_manager.cc:5495] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98 reported cstate change: term changed from 0 to 1, leader changed from <none> to aeb211ad2c004cd0a3a40e510f67ac98 (127.17.110.1). New cstate: current_term: 1 leader_uuid: "aeb211ad2c004cd0a3a40e510f67ac98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8cf58c0a4ff4d22933a471b69be07e9" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 38927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9819a0a0990421994d36137ef0bf904" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 33801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aeb211ad2c004cd0a3a40e510f67ac98" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 44935 } health_report { overall_health: HEALTHY } } }
W20260807 18:17:59.509341 6438 tablet.cc:2369] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:17:59.532796 6844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:59.527081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50788 (local address 127.17.110.1:44935)
0807 18:17:59.527326 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:59.527330 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:59.527351 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:17:59.527488 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:59.527492 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:59.527574 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:59.527725 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:17:59.528065 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.528884 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.530425 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.530870 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.530972 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:59.531527 (+ 555us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:17:59.532096 (+ 569us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:17:59.532354 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:59.532411 (+ 57us) server_negotiation.cc:299] Negotiation successful
0807 18:17:59.532503 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260807 18:17:59.540359 6621 tablet.cc:2369] T 97cae5f163e64242b62587b99ef8c14b P f9819a0a0990421994d36137ef0bf904: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:17:59.542762 6752 raft_consensus.cc:1261] T 97cae5f163e64242b62587b99ef8c14b P c8cf58c0a4ff4d22933a471b69be07e9 [term 1 FOLLOWER]: Refusing update from remote peer aeb211ad2c004cd0a3a40e510f67ac98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:17:59.542737 6566 raft_consensus.cc:1261] T 97cae5f163e64242b62587b99ef8c14b P f9819a0a0990421994d36137ef0bf904 [term 1 FOLLOWER]: Refusing update from remote peer aeb211ad2c004cd0a3a40e510f67ac98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:17:59.544064 6859 consensus_queue.cc:1035] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9819a0a0990421994d36137ef0bf904" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 33801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:17:59.544548 6852 consensus_queue.cc:1035] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8cf58c0a4ff4d22933a471b69be07e9" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 38927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:17:59.559303 6862 mvcc.cc:205] Tried to move back new op lower bound from 7315974879394127872 to 7315974879215542272. Current Snapshot: MvccSnapshot[applied={T|T < 7315974879394127872}]
I20260807 18:17:59.564272 6864 mvcc.cc:205] Tried to move back new op lower bound from 7315974879394127872 to 7315974879215542272. Current Snapshot: MvccSnapshot[applied={T|T < 7315974879394127872}]
I20260807 18:17:59.580052 6162 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:47426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:17:59.580412 6162 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:47426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:17:59.584589 6162 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906: Sending DeleteTablet for 3 replicas of tablet 97cae5f163e64242b62587b99ef8c14b
I20260807 18:17:59.586169 6343 tablet_service.cc:1514] Processing DeleteTablet for tablet 97cae5f163e64242b62587b99ef8c14b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:17:59 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:50774
I20260807 18:17:59.586323 6540 tablet_service.cc:1514] Processing DeleteTablet for tablet 97cae5f163e64242b62587b99ef8c14b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:17:59 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:41880
I20260807 18:17:59.586414 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 6219
I20260807 18:17:59.587656 6872 tablet_replica.cc:330] T 97cae5f163e64242b62587b99ef8c14b P f9819a0a0990421994d36137ef0bf904: stopping tablet replica
I20260807 18:17:59.588042 6726 tablet_service.cc:1514] Processing DeleteTablet for tablet 97cae5f163e64242b62587b99ef8c14b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:17:59 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:44996
I20260807 18:17:59.588384 6872 raft_consensus.cc:2229] T 97cae5f163e64242b62587b99ef8c14b P f9819a0a0990421994d36137ef0bf904 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:17:59.589018 6872 raft_consensus.cc:2258] T 97cae5f163e64242b62587b99ef8c14b P f9819a0a0990421994d36137ef0bf904 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:17:59.590044 6874 tablet_replica.cc:330] T 97cae5f163e64242b62587b99ef8c14b P c8cf58c0a4ff4d22933a471b69be07e9: stopping tablet replica
I20260807 18:17:59.590579 6874 raft_consensus.cc:2229] T 97cae5f163e64242b62587b99ef8c14b P c8cf58c0a4ff4d22933a471b69be07e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:17:59.591430 6874 raft_consensus.cc:2258] T 97cae5f163e64242b62587b99ef8c14b P c8cf58c0a4ff4d22933a471b69be07e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:17:59.592780 6873 tablet_replica.cc:330] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98: stopping tablet replica
I20260807 18:17:59.593281 6873 raft_consensus.cc:2229] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:17:59.594209 6874 ts_tablet_manager.cc:1905] T 97cae5f163e64242b62587b99ef8c14b P c8cf58c0a4ff4d22933a471b69be07e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:17:59.594198 6872 ts_tablet_manager.cc:1905] T 97cae5f163e64242b62587b99ef8c14b P f9819a0a0990421994d36137ef0bf904: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:17:59.595698 6873 raft_consensus.cc:2258] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:17:59.602517 6874 ts_tablet_manager.cc:1918] T 97cae5f163e64242b62587b99ef8c14b P c8cf58c0a4ff4d22933a471b69be07e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:17:59.602756 6874 log.cc:1196] T 97cae5f163e64242b62587b99ef8c14b P c8cf58c0a4ff4d22933a471b69be07e9: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/wal/wals/97cae5f163e64242b62587b99ef8c14b
I20260807 18:17:59.602803 6872 ts_tablet_manager.cc:1918] T 97cae5f163e64242b62587b99ef8c14b P f9819a0a0990421994d36137ef0bf904: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:17:59.603015 6872 log.cc:1196] T 97cae5f163e64242b62587b99ef8c14b P f9819a0a0990421994d36137ef0bf904: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/wal/wals/97cae5f163e64242b62587b99ef8c14b
I20260807 18:17:59.603441 6874 ts_tablet_manager.cc:1939] T 97cae5f163e64242b62587b99ef8c14b P c8cf58c0a4ff4d22933a471b69be07e9: Deleting consensus metadata
I20260807 18:17:59.603569 6872 ts_tablet_manager.cc:1939] T 97cae5f163e64242b62587b99ef8c14b P f9819a0a0990421994d36137ef0bf904: Deleting consensus metadata
I20260807 18:17:59.603753 6873 ts_tablet_manager.cc:1905] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:17:59.605531 6142 catalog_manager.cc:4841] TS f9819a0a0990421994d36137ef0bf904 (127.17.110.2:33801): tablet 97cae5f163e64242b62587b99ef8c14b (table test-table [id=83f6e771d9494bd5b54c82188b582ebb]) successfully deleted
I20260807 18:17:59.605609 6144 catalog_manager.cc:4841] TS c8cf58c0a4ff4d22933a471b69be07e9 (127.17.110.3:38927): tablet 97cae5f163e64242b62587b99ef8c14b (table test-table [id=83f6e771d9494bd5b54c82188b582ebb]) successfully deleted
I20260807 18:17:59.609409 6844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:59.587771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50800 (local address 127.17.110.1:44935)
0807 18:17:59.587992 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:59.588001 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:59.589418 (+ 1417us) server_negotiation.cc:407] Connection header received
0807 18:17:59.589511 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:59.589515 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:59.589614 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:59.589722 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:59.592256 (+ 2534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.593566 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.595209 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.595876 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.596053 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:59.600103 (+ 4050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:59.600131 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:59.600154 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:59.600213 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:59.605112 (+ 4899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:59.606228 (+ 1116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:59.606241 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:59.606249 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:59.606447 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:59.607126 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:59.607137 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:59.607139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:59.607678 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:59.608334 (+ 656us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:59.608724 (+ 390us) server_negotiation.cc:299] Negotiation successful
0807 18:17:59.609026 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:17:59.610447 6431 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:59.715631 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 6219
I20260807 18:17:59.744110 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 6443
W20260807 18:17:59.743988 6144 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.110.1:44935 (error 108)
I20260807 18:17:59.746073 6842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:59.745630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:44935 (local address 127.0.0.1:50804)
0807 18:17:59.745817 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:59.745923 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:44935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260807 18:17:59.746390 6144 catalog_manager.cc:4584] TS aeb211ad2c004cd0a3a40e510f67ac98 (127.17.110.1:44935): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 97cae5f163e64242b62587b99ef8c14b: Network error: Client connection negotiation failed: client connection to 127.17.110.1:44935: connect: Connection refused (error 111)
I20260807 18:17:59.760874 6840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:59.745049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41888 (local address 127.17.110.2:33801)
0807 18:17:59.745248 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:59.745255 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:59.745427 (+ 172us) server_negotiation.cc:407] Connection header received
0807 18:17:59.745643 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:59.745649 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:59.745741 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:59.745854 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:59.747520 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.748297 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.749791 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.750221 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.750370 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:59.753994 (+ 3624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:59.754016 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:59.754034 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:59.754091 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:59.757531 (+ 3440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:59.758527 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:59.758536 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:59.758539 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:59.758786 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:59.759256 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:59.759260 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:59.759263 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:59.759726 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:59.759994 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:59.760370 (+ 376us) server_negotiation.cc:299] Negotiation successful
0807 18:17:59.760574 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 18:17:59.761727 6613 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:59.800468 6842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:59.799836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:44935 (local address 127.0.0.1:50812)
0807 18:17:59.800092 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:59.800230 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:44935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260807 18:17:59.801962 6842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:59.801493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:44935 (local address 127.0.0.1:50824)
0807 18:17:59.801697 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:59.801793 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:44935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 18:17:59.858276 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 6443
I20260807 18:17:59.864343 6842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:59.863749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:44935 (local address 127.0.0.1:50840)
0807 18:17:59.864002 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:59.864171 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:44935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260807 18:17:59.865677 6842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:59.865261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:44935 (local address 127.0.0.1:50842)
0807 18:17:59.865459 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:59.865543 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:44935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 18:17:59.883217 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 6624
I20260807 18:17:59.907670 6831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:59.884126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45002 (local address 127.17.110.3:38927)
0807 18:17:59.884330 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:59.884334 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:59.884427 (+ 93us) server_negotiation.cc:407] Connection header received
0807 18:17:59.884579 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:59.884583 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:59.884667 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:59.884760 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:59.886273 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.887161 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.888706 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:59.889127 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:59.889238 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:59.901061 (+ 11823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:59.901081 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:59.901099 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:59.901146 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:59.904235 (+ 3089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:59.905185 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:59.905194 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:59.905197 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:59.905391 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:59.905925 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:59.905930 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:59.905932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:59.906392 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:59.906645 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:59.907123 (+ 478us) server_negotiation.cc:299] Negotiation successful
0807 18:17:59.907383 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:17:59.908413 6808 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:59.946583 6842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:59.945984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:44935 (local address 127.0.0.1:50852)
0807 18:17:59.946238 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:59.946361 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:44935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260807 18:17:59.948061 6842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:59.947546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:44935 (local address 127.0.0.1:50864)
0807 18:17:59.947797 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:59.947893 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:44935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
I20260807 18:18:00.004084 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 6624
I20260807 18:18:00.033325 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 6118
I20260807 18:18:00.034981 6210 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:18:00.176129 6842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:00.172806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:44935 (local address 127.0.0.1:50874)
0807 18:18:00.175477 (+ 2671us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:00.175705 (+ 228us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:44935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2344}
I20260807 18:18:00.176623 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 6118
I20260807 18:18:00.219092 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:42349
--webserver_interface=127.17.110.62
--webserver_port=38579
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:39359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:42349
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:18:00.663599 6881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:18:00.663980 6881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:18:00.664126 6881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:18:00.672840 6881 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:18:00.673015 6881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:18:00.673067 6881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:18:00.673206 6881 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:18:00.673307 6881 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:18:00.685129 6881 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:39359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:42349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:42349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=38579
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:18:00.686820 6881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:18:00.688725 6881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:18:00.706177 6887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:00.706326 6889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:00.706177 6886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:18:00.708997 6881 server_base.cc:1034] running on GCE node
I20260807 18:18:00.710453 6881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:18:00.712606 6881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:18:00.716214 6881 hybrid_clock.cc:648] HybridClock initialized: now 1786126680716100 us; error 109 us; skew 500 ppm
I20260807 18:18:00.716792 6881 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:18:00 dist-test-slave-39l8 krb5kdc[5321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126680, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:18:00.721478 6881 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:18:00.723830 6881 webserver.cc:460] Webserver started at http://127.17.110.62:38579/ using document root <none> and password file <none>
I20260807 18:18:00.724725 6881 fs_manager.cc:359] Metadata directory not provided
I20260807 18:18:00.724908 6881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:18:00.731838 6881 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:18:00.738101 6902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:00.740090 6881 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 18:18:00.740357 6881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "c9a45c2d73e04aeca5f641cd1d765906"
format_stamp: "Formatted at 2026-08-07 18:17:54 on dist-test-slave-39l8"
server_key: "db76b691220b44585db6aef8f9bb8743"
server_key_iv: "573da03a6a251cfc5a44111b20a339a6"
server_key_version: "encryptionkey@0"
I20260807 18:18:00.741659 6881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:18:00.761257 6881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:18:00.767925 6881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:18:00.768321 6881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:18:00.793354 6881 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:42349
I20260807 18:18:00.793375 6969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:42349 every 8 connection(s)
I20260807 18:18:00.795694 6881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:18:00.801733 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 6881
I20260807 18:18:00.803010 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.1:44935
--local_ip_for_outbound_sockets=127.17.110.1
--tserver_master_addrs=127.17.110.62:42349
--webserver_port=44317
--webserver_interface=127.17.110.1
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:39359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:18:00.806485 6971 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:18:00.813412 6971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906: Bootstrap starting.
I20260807 18:18:00.839823 6971 log.cc:824] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906: Log is configured to *not* fsync() on all Append() calls
I20260807 18:18:00.860209 6971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:18:00.860899 6971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906: Bootstrap complete.
I20260807 18:18:00.872900 6971 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9a45c2d73e04aeca5f641cd1d765906" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 42349 } }
I20260807 18:18:00.874095 6971 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9a45c2d73e04aeca5f641cd1d765906, State: Initialized, Role: FOLLOWER
I20260807 18:18:00.875146 6971 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9a45c2d73e04aeca5f641cd1d765906" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 42349 } }
I20260807 18:18:00.875478 6971 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:18:00.875631 6971 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:18:00.875864 6971 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [term 2 FOLLOWER]: Advancing to term 3
I20260807 18:18:00.880951 6971 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9a45c2d73e04aeca5f641cd1d765906" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 42349 } }
I20260807 18:18:00.881731 6971 leader_election.cc:302] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9a45c2d73e04aeca5f641cd1d765906; no voters:
I20260807 18:18:00.882649 6971 leader_election.cc:288] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 18:18:00.882891 6983 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 18:18:00.884469 6983 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [term 3 LEADER]: Becoming Leader. State: Replica: c9a45c2d73e04aeca5f641cd1d765906, State: Running, Role: LEADER
I20260807 18:18:00.885421 6983 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9a45c2d73e04aeca5f641cd1d765906" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 42349 } }
I20260807 18:18:00.886008 6971 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:18:00.889451 6985 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9a45c2d73e04aeca5f641cd1d765906. Latest consensus state: current_term: 3 leader_uuid: "c9a45c2d73e04aeca5f641cd1d765906" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9a45c2d73e04aeca5f641cd1d765906" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 42349 } } }
I20260807 18:18:00.890093 6985 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:00.890743 6984 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "c9a45c2d73e04aeca5f641cd1d765906" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9a45c2d73e04aeca5f641cd1d765906" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 42349 } } }
I20260807 18:18:00.891096 6984 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9a45c2d73e04aeca5f641cd1d765906 [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:00.899379 6998 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:18:00.911522 6998 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:18:00.913980 6998 catalog_manager.cc:1259] Loaded cluster ID: 2d92dac564b8453880b170c5b3a209fa
I20260807 18:18:00.914240 6998 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 18:18:00.917078 6998 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f78650769ad google::LogMessage::Fail() at ??:0
@ 0x7f786507988a google::LogMessage::SendToLog() at ??:0
@ 0x7f78650764de google::LogMessage::Flush() at ??:0
@ 0x7f7865077a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f787aaa9512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f787aaa6976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f787ab2e610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f787ab2e439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f7867cc2345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f7867cc591c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f7867cc5799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f7867ca0eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f786af7a6db start_thread at ??:0
@ 0x7f7863f1061f clone at ??:0
W20260807 18:18:01.280098 6981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:18:01.280498 6981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:18:01.280689 6981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:18:01.288712 6981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:18:01.288910 6981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:18:01.289109 6981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.1
I20260807 18:18:01.300289 6981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:39359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.1:44935
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.110.1
--webserver_port=44317
--tserver_master_addrs=127.17.110.62:42349
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.1
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:18:01.301800 6981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:18:01.303365 6981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:18:01.318315 7039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:01.318409 7042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:01.318442 7040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:18:01.319693 6981 server_base.cc:1034] running on GCE node
I20260807 18:18:01.320775 6981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:18:01.322122 6981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:18:01.323400 6981 hybrid_clock.cc:648] HybridClock initialized: now 1786126681323342 us; error 58 us; skew 500 ppm
I20260807 18:18:01.323807 6981 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:18:01 dist-test-slave-39l8 krb5kdc[5321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126681, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:18:01.327852 6981 init.cc:377] Logged in from keytab as kudu/127.17.110.1@KRBTEST.COM (short username kudu)
I20260807 18:18:01.329828 6981 webserver.cc:460] Webserver started at http://127.17.110.1:44317/ using document root <none> and password file <none>
I20260807 18:18:01.330679 6981 fs_manager.cc:359] Metadata directory not provided
I20260807 18:18:01.330909 6981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:18:01.337242 6981 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260807 18:18:01.341187 7050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:01.343180 6981 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 18:18:01.343425 6981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/wal
uuid: "aeb211ad2c004cd0a3a40e510f67ac98"
format_stamp: "Formatted at 2026-08-07 18:17:54 on dist-test-slave-39l8"
server_key: "a931ebb27a09180b4bc7c57e28a17834"
server_key_iv: "f602632313b312703a42ac4e7cda1a48"
server_key_version: "encryptionkey@0"
I20260807 18:18:01.344586 6981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:18:01.365522 6981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:18:01.371325 6981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:18:01.371698 6981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:18:01.373025 6981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:18:01.375415 7060 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260807 18:18:01.381951 7060 ts_tablet_manager.cc:1852] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260807 18:18:01.383208 7060 ts_tablet_manager.cc:1905] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:18:01.391754 7060 ts_tablet_manager.cc:1918] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:18:01.392051 7060 log.cc:1196] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/wal/wals/97cae5f163e64242b62587b99ef8c14b
I20260807 18:18:01.392604 7060 ts_tablet_manager.cc:1939] T 97cae5f163e64242b62587b99ef8c14b P aeb211ad2c004cd0a3a40e510f67ac98: Deleting consensus metadata
I20260807 18:18:01.393154 6981 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260807 18:18:01.393328 6981 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.018s user 0.000s sys 0.001s
I20260807 18:18:01.393503 6981 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:18:01.393610 6981 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:01.433974 6981 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.1:44935
I20260807 18:18:01.434175 7207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.1:44935 every 8 connection(s)
I20260807 18:18:01.435472 6981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
I20260807 18:18:01.441218 7211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:01.439002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.1:53723)
0807 18:18:01.439893 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:01.440199 (+ 306us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":330,"threads_started":1}
W20260807 18:18:01.442001 7208 heartbeater.cc:644] Failed to heartbeat to 127.17.110.62:42349 (0 consecutive failures): Network error: Failed to ping master at 127.17.110.62:42349: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
I20260807 18:18:01.443187 7211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:01.442680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.1:50351)
0807 18:18:01.442958 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:01.443051 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
I20260807 18:18:01.444303 7211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:01.443819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.1:60985)
0807 18:18:01.444048 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:01.444155 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260807 18:18:01.444486 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 6981
W20260807 18:18:01.444725 7208 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 18:18:01.445281 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.2:33801
--local_ip_for_outbound_sockets=127.17.110.2
--tserver_master_addrs=127.17.110.62:42349
--webserver_port=36373
--webserver_interface=127.17.110.2
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:39359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:18:01.821419 7212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:18:01.821777 7212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:18:01.821880 7212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:18:01.829589 7212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:18:01.829739 7212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:18:01.829859 7212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.2
I20260807 18:18:01.840176 7212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:39359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.2:33801
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.110.2
--webserver_port=36373
--tserver_master_addrs=127.17.110.62:42349
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.2
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:18:01.841706 7212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:18:01.843204 7212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:18:01.857656 7222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:01.857690 7226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:01.857717 7223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:18:01.859238 7212 server_base.cc:1034] running on GCE node
I20260807 18:18:01.860066 7212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:18:01.861254 7212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:18:01.862541 7212 hybrid_clock.cc:648] HybridClock initialized: now 1786126681862489 us; error 45 us; skew 500 ppm
I20260807 18:18:01.862978 7212 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:18:01 dist-test-slave-39l8 krb5kdc[5321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126681, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:18:01.870036 7212 init.cc:377] Logged in from keytab as kudu/127.17.110.2@KRBTEST.COM (short username kudu)
I20260807 18:18:01.871883 7212 webserver.cc:460] Webserver started at http://127.17.110.2:36373/ using document root <none> and password file <none>
I20260807 18:18:01.872679 7212 fs_manager.cc:359] Metadata directory not provided
I20260807 18:18:01.872798 7212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:18:01.879328 7212 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 18:18:01.883011 7239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:01.884809 7212 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:18:01.885067 7212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/wal
uuid: "f9819a0a0990421994d36137ef0bf904"
format_stamp: "Formatted at 2026-08-07 18:17:55 on dist-test-slave-39l8"
server_key: "8840004cfcbfe9919dab3cf453acf8e4"
server_key_iv: "9e5d3a254e8862dfe80a99555d35578d"
server_key_version: "encryptionkey@0"
I20260807 18:18:01.886377 7212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:18:01.912132 7212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:18:01.917765 7212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:18:01.918128 7212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:18:01.919471 7212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:18:01.921198 7212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:18:01.921312 7212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:01.921490 7212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:18:01.921582 7212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:01.963016 7212 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.2:33801
I20260807 18:18:01.963205 7399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.2:33801 every 8 connection(s)
I20260807 18:18:01.964713 7212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
I20260807 18:18:01.969154 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 7212
I20260807 18:18:01.970034 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.3:38927
--local_ip_for_outbound_sockets=127.17.110.3
--tserver_master_addrs=127.17.110.62:42349
--webserver_port=44603
--webserver_interface=127.17.110.3
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:39359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:18:01.971208 7403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:01.968244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.2:49229)
0807 18:18:01.969378 (+ 1134us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:01.969725 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":391,"threads_started":1}
W20260807 18:18:01.972182 7401 heartbeater.cc:644] Failed to heartbeat to 127.17.110.62:42349 (0 consecutive failures): Network error: Failed to ping master at 127.17.110.62:42349: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
I20260807 18:18:01.973407 7403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:01.972843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.2:33101)
0807 18:18:01.973081 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:01.973210 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260807 18:18:01.974768 7403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:01.974190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.2:36041)
0807 18:18:01.974411 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:01.974536 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260807 18:18:01.975150 7401 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:18:02.355823 7406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:18:02.356171 7406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:18:02.356357 7406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:18:02.364988 7406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:18:02.365134 7406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:18:02.365337 7406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.3
I20260807 18:18:02.375902 7406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:39359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.3:38927
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.110.3
--webserver_port=44603
--tserver_master_addrs=127.17.110.62:42349
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.3
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:18:02.377425 7406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:18:02.378971 7406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:18:02.395181 7414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:02.395177 7419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:02.395097 7415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:18:02.396445 7406 server_base.cc:1034] running on GCE node
I20260807 18:18:02.397207 7406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:18:02.398479 7406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:18:02.399772 7406 hybrid_clock.cc:648] HybridClock initialized: now 1786126682399709 us; error 68 us; skew 500 ppm
I20260807 18:18:02.400267 7406 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:18:02 dist-test-slave-39l8 krb5kdc[5321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126682, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:18:02.404421 7406 init.cc:377] Logged in from keytab as kudu/127.17.110.3@KRBTEST.COM (short username kudu)
I20260807 18:18:02.406314 7406 webserver.cc:460] Webserver started at http://127.17.110.3:44603/ using document root <none> and password file <none>
I20260807 18:18:02.407262 7406 fs_manager.cc:359] Metadata directory not provided
I20260807 18:18:02.407451 7406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:18:02.414011 7406 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:18:02.417441 7427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:02.419346 7406 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:18:02.419600 7406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/wal
uuid: "c8cf58c0a4ff4d22933a471b69be07e9"
format_stamp: "Formatted at 2026-08-07 18:17:55 on dist-test-slave-39l8"
server_key: "4a892bbf4046e5805f4eaa8133c2f917"
server_key_iv: "e389a16ce06e2a341ee4370d96a3cd3e"
server_key_version: "encryptionkey@0"
I20260807 18:18:02.420801 7406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:18:02.446467 7433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:02.445515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.1:33799)
0807 18:18:02.446063 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:02.446203 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":275,"threads_started":1}
I20260807 18:18:02.468119 7406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:18:02.473148 7406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:18:02.473497 7406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:18:02.474778 7406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:18:02.476418 7406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:18:02.476579 7406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:02.476745 7406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:18:02.476850 7406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:02.517295 7406 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.3:38927
I20260807 18:18:02.517400 7584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.3:38927 every 8 connection(s)
I20260807 18:18:02.519088 7406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
I20260807 18:18:02.524807 7590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:02.522641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.3:48089)
0807 18:18:02.523529 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:02.523793 (+ 264us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":324,"threads_started":1}
W20260807 18:18:02.525631 7585 heartbeater.cc:644] Failed to heartbeat to 127.17.110.62:42349 (0 consecutive failures): Network error: Failed to ping master at 127.17.110.62:42349: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
I20260807 18:18:02.526400 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 7406
I20260807 18:18:02.526945 7590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:02.526404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.3:48061)
0807 18:18:02.526660 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:02.526807 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260807 18:18:02.528374 7590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:02.527779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.3:58719)
0807 18:18:02.528005 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:02.528191 (+ 186us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260807 18:18:02.528728 7585 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 18:18:02.977008 7592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:02.975967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.2:36885)
0807 18:18:02.976598 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:02.976728 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":293,"threads_started":1}
I20260807 18:18:03.448479 7593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:03.447612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.1:41885)
0807 18:18:03.448214 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:03.448309 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":333,"threads_started":1}
I20260807 18:18:03.530428 7599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:03.529428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.3:33603)
0807 18:18:03.530021 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:03.530167 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":311,"threads_started":1}
I20260807 18:18:03.979333 7600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:03.978390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.2:48529)
0807 18:18:03.979027 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:03.979133 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":365,"threads_started":1}
I20260807 18:18:04.450512 7603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:04.449566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.1:60237)
0807 18:18:04.450221 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:04.450323 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":371,"threads_started":1}
I20260807 18:18:04.532629 7604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:04.531759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.3:42239)
0807 18:18:04.532303 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:04.532426 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":332,"threads_started":1}
I20260807 18:18:04.981278 7605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:04.980449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.2:38773)
0807 18:18:04.980954 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:04.981047 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":264,"threads_started":1}
I20260807 18:18:05.452688 7606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:05.451748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.1:36361)
0807 18:18:05.452333 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:05.452458 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":329,"threads_started":1}
I20260807 18:18:05.534400 7608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:05.533631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.3:52451)
0807 18:18:05.534139 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:05.534225 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":281,"threads_started":1}
I20260807 18:18:05.983188 7609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:05.982317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.2:59281)
0807 18:18:05.982863 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:05.982953 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":250,"threads_started":1}
I20260807 18:18:06.454921 7610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:06.453898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.1:35921)
0807 18:18:06.454489 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:06.454608 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":311,"threads_started":1}
I20260807 18:18:06.536537 7612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:06.535625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.3:55737)
0807 18:18:06.536213 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:06.536333 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":299,"threads_started":1}
I20260807 18:18:06.985241 7613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:06.984322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.2:58143)
0807 18:18:06.984901 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:06.984998 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":301,"threads_started":1}
I20260807 18:18:07.456919 7615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:07.456097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.1:52161)
0807 18:18:07.456611 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:07.456735 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":264,"threads_started":1}
I20260807 18:18:07.538866 7617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:07.537943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.3:37921)
0807 18:18:07.538523 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:07.538625 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":329,"threads_started":1}
I20260807 18:18:07.987537 7618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:07.986472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.2:52367)
0807 18:18:07.987229 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:07.987349 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":373,"threads_started":1}
I20260807 18:18:08.459324 7619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:08.458269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.1:33031)
0807 18:18:08.458969 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:08.459073 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":277,"threads_started":1}
I20260807 18:18:08.540863 7620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:08.540017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.3:45685)
0807 18:18:08.540567 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:08.540680 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":266,"threads_started":1}
I20260807 18:18:08.989781 7621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:08.988684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.2:33789)
0807 18:18:08.989346 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:08.989479 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":329,"threads_started":1}
I20260807 18:18:09.461462 7622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:09.460545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.1:34583)
0807 18:18:09.461151 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:09.461283 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":331,"threads_started":1}
I20260807 18:18:09.543160 7624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:09.542083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.3:53501)
0807 18:18:09.542791 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:09.542916 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":280,"threads_started":1}
I20260807 18:18:09.992074 7628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:09.990952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.2:55715)
0807 18:18:09.991632 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:09.991777 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":368,"threads_started":1}
I20260807 18:18:10.463577 7629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:10.462627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.1:60089)
0807 18:18:10.463276 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:10.463391 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":341,"threads_started":1}
I20260807 18:18:10.545250 7630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:10.544299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.3:39399)
0807 18:18:10.544892 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:10.544993 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":341,"threads_started":1}
I20260807 18:18:10.993935 7632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:10.993157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.2:59381)
0807 18:18:10.993664 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:10.993750 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":235,"threads_started":1}
I20260807 18:18:11.465430 7634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:11.464648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.1:45559)
0807 18:18:11.465125 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:11.465214 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":247,"threads_started":1}
I20260807 18:18:11.547120 7636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:11.546213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.3:46429)
0807 18:18:11.546818 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:11.546921 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":235,"threads_started":1}
I20260807 18:18:11.995649 7639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:11.994915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.2:56317)
0807 18:18:11.995389 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:11.995477 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":266,"threads_started":1}
I20260807 18:18:12.467475 7640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:12.466500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.1:43665)
0807 18:18:12.467157 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:12.467258 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":423,"threads_started":1}
I20260807 18:18:12.549093 7641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:12.548190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.3:50301)
0807 18:18:12.548780 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:12.548893 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":296,"threads_started":1}
I20260807 18:18:12.997609 7647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:12.996767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.2:42645)
0807 18:18:12.997280 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:12.997389 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":250,"threads_started":1}
I20260807 18:18:13.469517 7648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:13.468721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.1:52585)
0807 18:18:13.469248 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:13.469351 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":283,"threads_started":1}
I20260807 18:18:13.551323 7649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:13.550321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.3:37399)
0807 18:18:13.550995 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:13.551108 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":258,"threads_started":1}
I20260807 18:18:13.999791 7650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:13.998882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.2:49643)
0807 18:18:13.999447 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:13.999607 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":290,"threads_started":1}
I20260807 18:18:14.471562 7651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:14.470656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.1:37291)
0807 18:18:14.471253 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:14.471363 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":336,"threads_started":1}
I20260807 18:18:14.553258 7652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:14.552398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.3:51683)
0807 18:18:14.552992 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:14.553089 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":330,"threads_started":1}
I20260807 18:18:15.001983 7653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:15.001096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.2:54843)
0807 18:18:15.001664 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:15.001779 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":289,"threads_started":1}
I20260807 18:18:15.473554 7654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:15.472712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.1:52993)
0807 18:18:15.473267 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:15.473361 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":259,"threads_started":1}
I20260807 18:18:15.555354 7657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:15.554336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.3:57893)
0807 18:18:15.554986 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:15.555153 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":299,"threads_started":1}
I20260807 18:18:16.004225 7661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:16.003328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.2:38045)
0807 18:18:16.003950 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:16.004036 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":287,"threads_started":1}
I20260807 18:18:16.475888 7664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:16.474927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.1:38955)
0807 18:18:16.475552 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:16.475683 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":329,"threads_started":1}
I20260807 18:18:16.557130 7665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:16.556312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.3:52411)
0807 18:18:16.556846 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:16.556933 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":299,"threads_started":1}
I20260807 18:18:17.006450 7666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:17.005524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.2:54041)
0807 18:18:17.006059 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:17.006236 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":259,"threads_started":1}
I20260807 18:18:17.477919 7667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:17.477010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.1:54511)
0807 18:18:17.477652 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:17.477748 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":369,"threads_started":1}
I20260807 18:18:17.534771 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 6981
I20260807 18:18:17.559671 7669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:17.558582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42349 (local address 127.17.110.3:45519)
0807 18:18:17.559240 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:17.559430 (+ 190us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:42349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":330,"threads_started":1}
I20260807 18:18:17.561705 7668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:17.536250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33488 (local address 127.17.110.1:44935)
0807 18:18:17.537410 (+ 1160us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:17.537426 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:17.537473 (+ 47us) server_negotiation.cc:407] Connection header received
0807 18:18:17.539579 (+ 2106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:17.539591 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:17.540091 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:17.540441 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:17.542196 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:17.543783 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:17.545249 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:17.546077 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:17.546243 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:17.553580 (+ 7337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:17.553591 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:17.553613 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:17.553682 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:17.557363 (+ 3681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:17.558524 (+ 1161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:17.558571 (+ 47us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:17.558582 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:17.558924 (+ 342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:17.559499 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:17.559503 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:17.559505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:17.560240 (+ 735us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:17.560498 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:17.560940 (+ 442us) server_negotiation.cc:299] Negotiation successful
0807 18:18:17.561395 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":940,"thread_start_us":756,"threads_started":1}
I20260807 18:18:17.563217 7201 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:18:17.662891 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 6981
I20260807 18:18:17.692238 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 7212
I20260807 18:18:17.713613 7671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:17.693390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54412 (local address 127.17.110.2:33801)
0807 18:18:17.693931 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:17.693947 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:17.693988 (+ 41us) server_negotiation.cc:407] Connection header received
0807 18:18:17.695904 (+ 1916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:17.695918 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:17.696414 (+ 496us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:17.696819 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:17.698523 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:17.700168 (+ 1645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:17.701705 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:17.702501 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:17.702789 (+ 288us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:17.705942 (+ 3153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:17.705951 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:17.705970 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:17.706019 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:17.709571 (+ 3552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:17.710473 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:17.710482 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:17.710491 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:17.710700 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:17.711200 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:17.711206 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:17.711210 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:17.711843 (+ 633us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:17.712231 (+ 388us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:17.712777 (+ 546us) server_negotiation.cc:299] Negotiation successful
0807 18:18:17.713276 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":259,"threads_started":1}
I20260807 18:18:17.715349 7393 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:18:17.804000 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 7212
I20260807 18:18:17.830585 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 7406
I20260807 18:18:17.850173 7673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:17.831659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60388 (local address 127.17.110.3:38927)
0807 18:18:17.832326 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:17.832342 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:17.832388 (+ 46us) server_negotiation.cc:407] Connection header received
0807 18:18:17.834296 (+ 1908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:17.834309 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:17.834949 (+ 640us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:17.835273 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:17.836750 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:17.838035 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:17.839378 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:17.840069 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:17.840211 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:17.843123 (+ 2912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:17.843134 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:17.843155 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:17.843204 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:17.846455 (+ 3251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:17.847321 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:17.847330 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:17.847338 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:17.847581 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:17.847984 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:17.847987 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:17.847989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:17.848655 (+ 666us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:17.848935 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:17.849396 (+ 461us) server_negotiation.cc:299] Negotiation successful
0807 18:18:17.849819 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":326,"threads_started":1}
I20260807 18:18:17.851859 7580 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:18:17.939726 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 7406
2026-08-07T18:18:17Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (26036 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:18:18 dist-test-slave-39l8 krb5kdc[7680](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:18:18 dist-test-slave-39l8 krb5kdc[7680](info): set up 2 sockets
Aug 07 18:18:18 dist-test-slave-39l8 krb5kdc[7680](info): commencing operation
krb5kdc: starting...
W20260807 18:18:19.301728 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.314s user 0.013s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:18:19 dist-test-slave-39l8 krb5kdc[7680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:18:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:18:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.110.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
I20260807 18:18:19.632800 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:45375
--webserver_interface=127.17.110.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:46703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:45375
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:18:19.997058 7701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:18:19.997399 7701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:18:19.997527 7701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:18:20.005564 7701 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 18:18:20.005702 7701 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 18:18:20.005775 7701 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 18:18:20.005864 7701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:18:20.005959 7701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:18:20.006078 7701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:18:20.006151 7701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:18:20.006244 7701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:18:20.020347 7701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:46703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:45375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:45375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:18:20.022120 7701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:18:20.023715 7701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:18:20.037575 7709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:20.037673 7711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:20.037650 7708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:18:20.037940 7701 server_base.cc:1034] running on GCE node
I20260807 18:18:20.039459 7701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:18:20.041363 7701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:18:20.042644 7701 hybrid_clock.cc:648] HybridClock initialized: now 1786126700042577 us; error 44 us; skew 500 ppm
I20260807 18:18:20.043074 7701 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:18:20 dist-test-slave-39l8 krb5kdc[7680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126700, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:18:20.046691 7701 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:18:20.048491 7701 webserver.cc:460] Webserver started at http://127.17.110.62:44325/ using document root <none> and password file <none>
I20260807 18:18:20.049361 7701 fs_manager.cc:359] Metadata directory not provided
I20260807 18:18:20.049515 7701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:18:20.049896 7701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:18:20.053469 7701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "7a3b1cd4083644cabeef6647a97bc5af"
format_stamp: "Formatted at 2026-08-07 18:18:20 on dist-test-slave-39l8"
server_key: "c4c13ef066df83212eb28a8223bc959a"
server_key_iv: "ab9f0012e71f5d53b3a51ec2d01a39c1"
server_key_version: "encryptionkey@0"
I20260807 18:18:20.054400 7701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "7a3b1cd4083644cabeef6647a97bc5af"
format_stamp: "Formatted at 2026-08-07 18:18:20 on dist-test-slave-39l8"
server_key: "c4c13ef066df83212eb28a8223bc959a"
server_key_iv: "ab9f0012e71f5d53b3a51ec2d01a39c1"
server_key_version: "encryptionkey@0"
I20260807 18:18:20.060426 7701 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260807 18:18:20.064489 7721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:20.066093 7701 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:18:20.066347 7701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "7a3b1cd4083644cabeef6647a97bc5af"
format_stamp: "Formatted at 2026-08-07 18:18:20 on dist-test-slave-39l8"
server_key: "c4c13ef066df83212eb28a8223bc959a"
server_key_iv: "ab9f0012e71f5d53b3a51ec2d01a39c1"
server_key_version: "encryptionkey@0"
I20260807 18:18:20.066617 7701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:18:20.082670 7701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:18:20.087672 7701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:18:20.088009 7701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:18:20.110795 7701 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:45375
I20260807 18:18:20.110834 7791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:45375 every 8 connection(s)
I20260807 18:18:20.112423 7701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:18:20.115307 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 7701
I20260807 18:18:20.115581 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:18:20.116482 17848 external_mini_cluster.cc:1428] Setting key eeeb14da4cf5a90b0498a0a80996bfb0
I20260807 18:18:20.117491 7792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:18:20.134754 7792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af: Bootstrap starting.
Aug 07 18:18:20 dist-test-slave-39l8 krb5kdc[7680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:18:20.138521 7792 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af: Neither blocks nor log segments found. Creating new log.
I20260807 18:18:20.139947 7792 log.cc:824] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af: Log is configured to *not* fsync() on all Append() calls
I20260807 18:18:20.143489 7792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af: No bootstrap required, opened a new log
I20260807 18:18:20.146075 7800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:20.119001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51632 (local address 127.17.110.62:45375)
0807 18:18:20.119874 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:20.119888 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:20.119934 (+ 46us) server_negotiation.cc:407] Connection header received
0807 18:18:20.123211 (+ 3277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:20.123250 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:20.123932 (+ 682us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:20.124502 (+ 570us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:20.126771 (+ 2269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:20.128338 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:20.130493 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:20.131469 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:20.131632 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:20.137222 (+ 5590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:20.137231 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:20.137247 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:20.137299 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:20.141185 (+ 3886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:20.142076 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:20.142085 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:20.142092 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:20.142295 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:20.142787 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:20.142793 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:20.142796 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:20.143446 (+ 650us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:20.143812 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:20.144367 (+ 555us) server_negotiation.cc:299] Negotiation successful
0807 18:18:20.144881 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":383,"threads_started":1}
I20260807 18:18:20.151891 7792 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3b1cd4083644cabeef6647a97bc5af" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45375 } }
I20260807 18:18:20.152401 7792 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:18:20.152523 7792 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a3b1cd4083644cabeef6647a97bc5af, State: Initialized, Role: FOLLOWER
I20260807 18:18:20.153198 7792 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3b1cd4083644cabeef6647a97bc5af" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45375 } }
I20260807 18:18:20.153457 7792 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:18:20.153617 7792 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:18:20.153812 7792 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:18:20.156615 7792 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3b1cd4083644cabeef6647a97bc5af" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45375 } }
I20260807 18:18:20.157153 7792 leader_election.cc:302] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a3b1cd4083644cabeef6647a97bc5af; no voters:
I20260807 18:18:20.157918 7792 leader_election.cc:288] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:18:20.158077 7802 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:18:20.158934 7802 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [term 1 LEADER]: Becoming Leader. State: Replica: 7a3b1cd4083644cabeef6647a97bc5af, State: Running, Role: LEADER
I20260807 18:18:20.159560 7802 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3b1cd4083644cabeef6647a97bc5af" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45375 } }
I20260807 18:18:20.160600 7792 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:18:20.163123 7803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a3b1cd4083644cabeef6647a97bc5af. Latest consensus state: current_term: 1 leader_uuid: "7a3b1cd4083644cabeef6647a97bc5af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3b1cd4083644cabeef6647a97bc5af" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45375 } } }
I20260807 18:18:20.163153 7804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a3b1cd4083644cabeef6647a97bc5af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3b1cd4083644cabeef6647a97bc5af" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45375 } } }
I20260807 18:18:20.163689 7804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:20.163686 7803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:20.165561 7810 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:18:20.172679 7810 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:18:20.182469 7810 catalog_manager.cc:1347] Generated new cluster ID: ab3bf186d2524cd989e7a485c948352c
I20260807 18:18:20.182624 7810 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:18:20.197206 7810 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:18:20.199505 7810 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:18:20.207301 7810 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af: Generated new TSK 0
I20260807 18:18:20.208550 7810 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.110.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
I20260807 18:18:20.288130 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.1:0
--local_ip_for_outbound_sockets=127.17.110.1
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:45375
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:46703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:18:20.695693 7834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:18:20.696005 7834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:18:20.696164 7834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:18:20.703791 7834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:18:20.703923 7834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:18:20.704166 7834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.1
I20260807 18:18:20.714895 7834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:46703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:45375
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.1
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:18:20.716255 7834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:18:20.717669 7834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:18:20.732052 7851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:20.732012 7847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:20.732095 7846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:18:20.732980 7834 server_base.cc:1034] running on GCE node
I20260807 18:18:20.733685 7834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:18:20.734810 7834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:18:20.736131 7834 hybrid_clock.cc:648] HybridClock initialized: now 1786126700736057 us; error 55 us; skew 500 ppm
I20260807 18:18:20.736550 7834 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:18:20 dist-test-slave-39l8 krb5kdc[7680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126700, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:18:20.740435 7834 init.cc:377] Logged in from keytab as kudu/127.17.110.1@KRBTEST.COM (short username kudu)
I20260807 18:18:20.742293 7834 webserver.cc:460] Webserver started at http://127.17.110.1:33745/ using document root <none> and password file <none>
I20260807 18:18:20.743151 7834 fs_manager.cc:359] Metadata directory not provided
I20260807 18:18:20.743310 7834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:18:20.743646 7834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:18:20.746865 7834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/data/instance:
uuid: "829552e4c23a43349356c4c34c784c62"
format_stamp: "Formatted at 2026-08-07 18:18:20 on dist-test-slave-39l8"
server_key: "dcc5621c19c74d9c7e41f6c9fd89a00f"
server_key_iv: "29553daaa980ebbd0043c8251d86ee53"
server_key_version: "encryptionkey@0"
I20260807 18:18:20.747682 7834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance:
uuid: "829552e4c23a43349356c4c34c784c62"
format_stamp: "Formatted at 2026-08-07 18:18:20 on dist-test-slave-39l8"
server_key: "dcc5621c19c74d9c7e41f6c9fd89a00f"
server_key_iv: "29553daaa980ebbd0043c8251d86ee53"
server_key_version: "encryptionkey@0"
I20260807 18:18:20.753399 7834 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 18:18:20.757263 7859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:20.758694 7834 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 18:18:20.758949 7834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/wal
uuid: "829552e4c23a43349356c4c34c784c62"
format_stamp: "Formatted at 2026-08-07 18:18:20 on dist-test-slave-39l8"
server_key: "dcc5621c19c74d9c7e41f6c9fd89a00f"
server_key_iv: "29553daaa980ebbd0043c8251d86ee53"
server_key_version: "encryptionkey@0"
I20260807 18:18:20.759187 7834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:18:20.772922 7834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:18:20.777820 7834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:18:20.778156 7834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:18:20.779311 7834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:18:20.780855 7834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:18:20.780969 7834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:20.781116 7834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:18:20.781188 7834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:20.818270 7834 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.1:34987
I20260807 18:18:20.818322 8016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.1:34987 every 8 connection(s)
I20260807 18:18:20.819700 7834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
I20260807 18:18:20.826339 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 7834
I20260807 18:18:20.826639 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance
I20260807 18:18:20.827644 17848 external_mini_cluster.cc:1428] Setting key f6ef483633ed67b6546bdce3d7a38a25
Aug 07 18:18:20 dist-test-slave-39l8 krb5kdc[7680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126700, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:18:20.844854 8020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:20.822513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:45767 (local address 127.17.110.62:45375)
0807 18:18:20.823046 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:20.823053 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:20.824111 (+ 1058us) server_negotiation.cc:407] Connection header received
0807 18:18:20.826509 (+ 2398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:20.826517 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:20.826612 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:20.826778 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:20.829141 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:20.829946 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:20.831778 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:20.832659 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:20.832853 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:20.837645 (+ 4792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:20.837652 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:20.837656 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:20.837697 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:20.840545 (+ 2848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:20.841680 (+ 1135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:20.841685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:20.841687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:20.841848 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:20.842632 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:20.842639 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:20.842643 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:20.843065 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:20.843264 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:20.844142 (+ 878us) server_negotiation.cc:299] Negotiation successful
0807 18:18:20.844430 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":295,"threads_started":1}
I20260807 18:18:20.845978 8021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:20.823166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.1:45767)
0807 18:18:20.823905 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:20.823955 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:20.826083 (+ 2128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:20.827111 (+ 1028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:20.827125 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:20.828110 (+ 985us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:18:20.828904 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:20.828927 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:20.830167 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:20.830172 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:20.831600 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:20.831615 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:20.832962 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:20.832969 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:20.833191 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:20.833930 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:18:20.833956 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:18:20.837327 (+ 3371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:18:20.840744 (+ 3417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:20.840758 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:20.840782 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:20.841523 (+ 741us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:20.841972 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:20.841980 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:20.841987 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:20.842341 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:20.843243 (+ 902us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:18:20.843253 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:18:20.843684 (+ 431us) client_negotiation.cc:769] Sending connection context
0807 18:18:20.844004 (+ 320us) client_negotiation.cc:240] Negotiation successful
0807 18:18:20.844666 (+ 662us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":288,"threads_started":1}
I20260807 18:18:20.847839 8017 heartbeater.cc:347] Connected to a master server at 127.17.110.62:45375
I20260807 18:18:20.848229 8017 heartbeater.cc:464] Registering TS with master...
I20260807 18:18:20.849045 8017 heartbeater.cc:511] Master 127.17.110.62:45375 requested a full tablet report, sending...
I20260807 18:18:20.851665 7745 ts_manager.cc:194] Registered new tserver with Master: 829552e4c23a43349356c4c34c784c62 (127.17.110.1:34987)
I20260807 18:18:20.854007 7745 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.1@KRBTEST.COM'} at 127.17.110.1:45767
WARNING: no policy specified for kudu/127.17.110.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
I20260807 18:18:20.898239 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.2:0
--local_ip_for_outbound_sockets=127.17.110.2
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:45375
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:46703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:18:21.287843 8025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:18:21.288177 8025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:18:21.288285 8025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:18:21.296170 8025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:18:21.296316 8025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:18:21.296437 8025 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.2
I20260807 18:18:21.306996 8025 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:46703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:45375
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.2
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:18:21.308393 8025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:18:21.309832 8025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:18:21.324376 8033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:21.324388 8037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:21.324548 8034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:18:21.331676 8025 server_base.cc:1034] running on GCE node
I20260807 18:18:21.332557 8025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:18:21.333724 8025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:18:21.334962 8025 hybrid_clock.cc:648] HybridClock initialized: now 1786126701334904 us; error 53 us; skew 500 ppm
I20260807 18:18:21.335341 8025 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:18:21 dist-test-slave-39l8 krb5kdc[7680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126701, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:18:21.339111 8025 init.cc:377] Logged in from keytab as kudu/127.17.110.2@KRBTEST.COM (short username kudu)
I20260807 18:18:21.340938 8025 webserver.cc:460] Webserver started at http://127.17.110.2:43341/ using document root <none> and password file <none>
I20260807 18:18:21.341737 8025 fs_manager.cc:359] Metadata directory not provided
I20260807 18:18:21.341893 8025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:18:21.342242 8025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:18:21.345753 8025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/data/instance:
uuid: "775ee470852342518a03a533889cc550"
format_stamp: "Formatted at 2026-08-07 18:18:21 on dist-test-slave-39l8"
server_key: "345d8a4e5013cf01743d32339088b2d4"
server_key_iv: "66e985761c60acea9673dba49f81f3ff"
server_key_version: "encryptionkey@0"
I20260807 18:18:21.346619 8025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance:
uuid: "775ee470852342518a03a533889cc550"
format_stamp: "Formatted at 2026-08-07 18:18:21 on dist-test-slave-39l8"
server_key: "345d8a4e5013cf01743d32339088b2d4"
server_key_iv: "66e985761c60acea9673dba49f81f3ff"
server_key_version: "encryptionkey@0"
I20260807 18:18:21.353011 8025 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260807 18:18:21.357746 8047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:21.359508 8025 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 18:18:21.359735 8025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/wal
uuid: "775ee470852342518a03a533889cc550"
format_stamp: "Formatted at 2026-08-07 18:18:21 on dist-test-slave-39l8"
server_key: "345d8a4e5013cf01743d32339088b2d4"
server_key_iv: "66e985761c60acea9673dba49f81f3ff"
server_key_version: "encryptionkey@0"
I20260807 18:18:21.359998 8025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:18:21.381628 8025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:18:21.386834 8025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:18:21.387184 8025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:18:21.388330 8025 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:18:21.390134 8025 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:18:21.390264 8025 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:21.390435 8025 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:18:21.390513 8025 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:21.429541 8025 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.2:38123
I20260807 18:18:21.429616 8213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.2:38123 every 8 connection(s)
I20260807 18:18:21.431114 8025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
I20260807 18:18:21.433173 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 8025
I20260807 18:18:21.433395 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance
I20260807 18:18:21.433938 17848 external_mini_cluster.cc:1428] Setting key 1e77a0647a39e52b5e171819baa298fe
Aug 07 18:18:21 dist-test-slave-39l8 krb5kdc[7680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126701, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:18:21.455411 8218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:21.434471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:39123 (local address 127.17.110.62:45375)
0807 18:18:21.435024 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:21.435031 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:21.436215 (+ 1184us) server_negotiation.cc:407] Connection header received
0807 18:18:21.438309 (+ 2094us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:21.438314 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:21.438415 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:21.438541 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:21.440302 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:21.441041 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:21.442853 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:21.443301 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:21.443449 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:21.447761 (+ 4312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:21.447766 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:21.447770 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:21.447810 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:21.451060 (+ 3250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:21.452379 (+ 1319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:21.452384 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:21.452386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:21.452554 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:21.453191 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:21.453195 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:21.453197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:21.453541 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:21.453681 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:21.454857 (+ 1176us) server_negotiation.cc:299] Negotiation successful
0807 18:18:21.455109 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":287,"threads_started":1}
I20260807 18:18:21.457062 8219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:21.435158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.2:39123)
0807 18:18:21.436046 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:21.436107 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:21.437983 (+ 1876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:21.438871 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:21.438880 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:21.439478 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:18:21.440105 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:21.440124 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:21.441235 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:21.441247 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:21.442625 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:21.442641 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:21.443678 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:21.443692 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:21.443972 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:21.444741 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:18:21.444771 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:18:21.447445 (+ 2674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:18:21.451306 (+ 3861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:21.451321 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:21.451346 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:21.452164 (+ 818us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:21.452723 (+ 559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:21.452731 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:21.452738 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:21.453040 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:21.453703 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:18:21.453715 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:18:21.454250 (+ 535us) client_negotiation.cc:769] Sending connection context
0807 18:18:21.454602 (+ 352us) client_negotiation.cc:240] Negotiation successful
0807 18:18:21.455318 (+ 716us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":401,"threads_started":1}
I20260807 18:18:21.458950 8214 heartbeater.cc:347] Connected to a master server at 127.17.110.62:45375
I20260807 18:18:21.459335 8214 heartbeater.cc:464] Registering TS with master...
I20260807 18:18:21.460210 8214 heartbeater.cc:511] Master 127.17.110.62:45375 requested a full tablet report, sending...
I20260807 18:18:21.462143 7745 ts_manager.cc:194] Registered new tserver with Master: 775ee470852342518a03a533889cc550 (127.17.110.2:38123)
I20260807 18:18:21.463608 7745 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.2@KRBTEST.COM'} at 127.17.110.2:39123
WARNING: no policy specified for kudu/127.17.110.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
I20260807 18:18:21.509994 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.3:0
--local_ip_for_outbound_sockets=127.17.110.3
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:45375
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:46703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:18:21.857457 8017 heartbeater.cc:503] Master 127.17.110.62:45375 was elected leader, sending a full tablet report...
W20260807 18:18:21.900134 8223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:18:21.900470 8223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:18:21.900645 8223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:18:21.908270 8223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:18:21.908416 8223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:18:21.908609 8223 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.3
I20260807 18:18:21.919060 8223 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:46703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:45375
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.3
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:18:21.920575 8223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:18:21.922001 8223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:18:21.936779 8235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:21.936651 8232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:21.936822 8233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:18:21.938791 8223 server_base.cc:1034] running on GCE node
I20260807 18:18:21.939585 8223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:18:21.940759 8223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:18:21.942060 8223 hybrid_clock.cc:648] HybridClock initialized: now 1786126701941985 us; error 55 us; skew 500 ppm
I20260807 18:18:21.942461 8223 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:18:21 dist-test-slave-39l8 krb5kdc[7680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126701, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:18:21.946846 8223 init.cc:377] Logged in from keytab as kudu/127.17.110.3@KRBTEST.COM (short username kudu)
I20260807 18:18:21.948776 8223 webserver.cc:460] Webserver started at http://127.17.110.3:43873/ using document root <none> and password file <none>
I20260807 18:18:21.949635 8223 fs_manager.cc:359] Metadata directory not provided
I20260807 18:18:21.949815 8223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:18:21.950179 8223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:18:21.953759 8223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/data/instance:
uuid: "533163cd8b404049ae59946e598492fa"
format_stamp: "Formatted at 2026-08-07 18:18:21 on dist-test-slave-39l8"
server_key: "550f862ace80c150582d7ab6bead54f5"
server_key_iv: "ce733b93103111465bb94eabed1ccfd6"
server_key_version: "encryptionkey@0"
I20260807 18:18:21.954825 8223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance:
uuid: "533163cd8b404049ae59946e598492fa"
format_stamp: "Formatted at 2026-08-07 18:18:21 on dist-test-slave-39l8"
server_key: "550f862ace80c150582d7ab6bead54f5"
server_key_iv: "ce733b93103111465bb94eabed1ccfd6"
server_key_version: "encryptionkey@0"
I20260807 18:18:21.961645 8223 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260807 18:18:21.966352 8244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:21.968331 8223 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:18:21.968565 8223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/wal
uuid: "533163cd8b404049ae59946e598492fa"
format_stamp: "Formatted at 2026-08-07 18:18:21 on dist-test-slave-39l8"
server_key: "550f862ace80c150582d7ab6bead54f5"
server_key_iv: "ce733b93103111465bb94eabed1ccfd6"
server_key_version: "encryptionkey@0"
I20260807 18:18:21.968838 8223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:18:21.986115 8223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:18:21.991503 8223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:18:21.991906 8223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:18:21.993012 8223 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:18:21.994494 8223 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:18:21.994582 8223 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:21.994788 8223 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:18:21.994846 8223 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:22.039081 8223 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.3:36227
I20260807 18:18:22.039171 8406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.3:36227 every 8 connection(s)
I20260807 18:18:22.040512 8223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
I20260807 18:18:22.044581 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 8223
I20260807 18:18:22.044849 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance
I20260807 18:18:22.045614 17848 external_mini_cluster.cc:1428] Setting key 7f25ac00e4aaeb7a7207509c94877edf
Aug 07 18:18:22 dist-test-slave-39l8 krb5kdc[7680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126701, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:18:22.065220 8412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:22.043485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:33217 (local address 127.17.110.62:45375)
0807 18:18:22.044156 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:22.044165 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:22.045305 (+ 1140us) server_negotiation.cc:407] Connection header received
0807 18:18:22.048363 (+ 3058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:22.048367 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:22.048471 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:22.048612 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:22.050875 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.051763 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.053459 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.053894 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.053998 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:22.058002 (+ 4004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:22.058008 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:22.058011 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:22.058049 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:22.061476 (+ 3427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:22.062454 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:22.062461 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:22.062465 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:22.062666 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:22.063173 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:22.063181 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:22.063184 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:22.063571 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:22.063742 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:22.064531 (+ 789us) server_negotiation.cc:299] Negotiation successful
0807 18:18:22.064787 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":292,"threads_started":1}
I20260807 18:18:22.066280 8413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:22.044120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.3:33217)
0807 18:18:22.045075 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:22.045132 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:22.047959 (+ 2827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:22.048932 (+ 973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:22.048941 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:22.049804 (+ 863us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:18:22.050512 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:22.050533 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.051959 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.051964 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:22.053296 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:22.053310 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.054120 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.054125 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:22.054340 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:22.055147 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:18:22.055172 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:18:22.057725 (+ 2553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:18:22.061645 (+ 3920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:22.061654 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:22.061677 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:22.062322 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:22.062823 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:22.062828 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:22.062832 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:22.063048 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:22.063730 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:18:22.063739 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:18:22.064128 (+ 389us) client_negotiation.cc:769] Sending connection context
0807 18:18:22.064458 (+ 330us) client_negotiation.cc:240] Negotiation successful
0807 18:18:22.065029 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":338,"threads_started":1}
I20260807 18:18:22.067998 8408 heartbeater.cc:347] Connected to a master server at 127.17.110.62:45375
I20260807 18:18:22.068313 8408 heartbeater.cc:464] Registering TS with master...
I20260807 18:18:22.069069 8408 heartbeater.cc:511] Master 127.17.110.62:45375 requested a full tablet report, sending...
I20260807 18:18:22.070693 7745 ts_manager.cc:194] Registered new tserver with Master: 533163cd8b404049ae59946e598492fa (127.17.110.3:36227)
I20260807 18:18:22.071791 7745 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.3@KRBTEST.COM'} at 127.17.110.3:33217
I20260807 18:18:22.075170 17848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:18:22.095139 8412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:22.080306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37016 (local address 127.17.110.62:45375)
0807 18:18:22.080560 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:22.080568 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:22.080849 (+ 281us) server_negotiation.cc:407] Connection header received
0807 18:18:22.081109 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:22.081113 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:22.081187 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:22.081307 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:22.083147 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.083913 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.085467 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.085893 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.086042 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:22.088826 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:22.088832 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:22.088835 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:22.088881 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:22.091908 (+ 3027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:22.092774 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:22.092781 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:22.092784 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:22.092946 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:22.093387 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:22.093399 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:22.093402 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:22.093774 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:22.094066 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:22.094467 (+ 401us) server_negotiation.cc:299] Negotiation successful
0807 18:18:22.094770 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 18:18:22.100291 7745 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37016:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:18:22.108556 7745 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:18:22.144912 8429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:22.124419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53080 (local address 127.17.110.2:38123)
0807 18:18:22.134351 (+ 9932us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:22.134360 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:22.135592 (+ 1232us) server_negotiation.cc:407] Connection header received
0807 18:18:22.135746 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:22.135757 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:22.136039 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:22.136251 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:18:22.137406 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.138997 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.141696 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.143363 (+ 1667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.143512 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:22.143736 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:22.144363 (+ 627us) server_negotiation.cc:299] Negotiation successful
0807 18:18:22.144534 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9760,"thread_start_us":9617,"threads_started":1}
I20260807 18:18:22.144964 8428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:22.123824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:38123 (local address 127.0.0.1:53080)
0807 18:18:22.135036 (+ 11212us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:22.135093 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:22.135349 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:22.136563 (+ 1214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:22.136571 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:22.136611 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:18:22.137172 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:22.137186 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.139143 (+ 1957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.139150 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:22.141472 (+ 2322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:22.141487 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.143600 (+ 2113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.143607 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:22.143980 (+ 373us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:22.144055 (+ 75us) client_negotiation.cc:769] Sending connection context
0807 18:18:22.144293 (+ 238us) client_negotiation.cc:240] Negotiation successful
0807 18:18:22.144525 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10928,"mutex_wait_us":90,"thread_start_us":10741,"threads_started":1}
I20260807 18:18:22.145198 8431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:22.125357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43848 (local address 127.17.110.3:36227)
0807 18:18:22.135586 (+ 10229us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:22.135595 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:22.136170 (+ 575us) server_negotiation.cc:407] Connection header received
0807 18:18:22.136942 (+ 772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:22.136954 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:22.137250 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:22.137460 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:18:22.138650 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.140258 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.142685 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.143822 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.143964 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:22.144231 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:22.144561 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 18:18:22.144796 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10043,"thread_start_us":9769,"threads_started":1}
I20260807 18:18:22.146032 8430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:22.124573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:36227 (local address 127.0.0.1:43848)
0807 18:18:22.136017 (+ 11444us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:22.136037 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:22.136186 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:22.137761 (+ 1575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:22.137766 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:22.137796 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:18:22.138382 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:22.138396 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.140421 (+ 2025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.140429 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:22.142516 (+ 2087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:22.142528 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.144067 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.144073 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:22.144264 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:22.144308 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 18:18:22.145477 (+ 1169us) client_negotiation.cc:240] Negotiation successful
0807 18:18:22.145623 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11324,"mutex_wait_us":3,"thread_start_us":11184,"threads_started":1}
I20260807 18:18:22.148521 8435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:22.136730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:34987 (local address 127.0.0.1:52906)
0807 18:18:22.138342 (+ 1612us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:22.138359 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:22.138518 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:22.139576 (+ 1058us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:22.139581 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:22.139608 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:18:22.140063 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:22.140071 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.142167 (+ 2096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.142174 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:22.144432 (+ 2258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:22.144448 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.147631 (+ 3183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.147639 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:22.147822 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:22.147855 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 18:18:22.147968 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 18:18:22.148126 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1484,"mutex_wait_us":45,"thread_start_us":1304,"threads_started":1}
I20260807 18:18:22.148530 8434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:22.135971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52906 (local address 127.17.110.1:34987)
0807 18:18:22.136724 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:22.136732 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:22.138480 (+ 1748us) server_negotiation.cc:407] Connection header received
0807 18:18:22.139217 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:22.139223 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:22.139389 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:22.139593 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:18:22.140261 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.141931 (+ 1670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.144619 (+ 2688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.147043 (+ 2424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.147189 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:22.147430 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:22.148046 (+ 616us) server_negotiation.cc:299] Negotiation successful
0807 18:18:22.148224 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":417,"threads_started":1}
I20260807 18:18:22.149277 8113 tablet_service.cc:1467] Processing CreateTablet for tablet 472bb383ff0c430aa1d54864fe380fe7 (DEFAULT_TABLE table=test-table [id=af19e6e9c25544e8b070e14794d9576a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:18:22.149893 8313 tablet_service.cc:1467] Processing CreateTablet for tablet 472bb383ff0c430aa1d54864fe380fe7 (DEFAULT_TABLE table=test-table [id=af19e6e9c25544e8b070e14794d9576a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:18:22.150838 8113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 472bb383ff0c430aa1d54864fe380fe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:18:22.151165 8313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 472bb383ff0c430aa1d54864fe380fe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:18:22.152606 7925 tablet_service.cc:1467] Processing CreateTablet for tablet 472bb383ff0c430aa1d54864fe380fe7 (DEFAULT_TABLE table=test-table [id=af19e6e9c25544e8b070e14794d9576a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:18:22.154066 7925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 472bb383ff0c430aa1d54864fe380fe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:18:22.163650 8436 tablet_bootstrap.cc:492] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa: Bootstrap starting.
I20260807 18:18:22.167618 8437 tablet_bootstrap.cc:492] T 472bb383ff0c430aa1d54864fe380fe7 P 775ee470852342518a03a533889cc550: Bootstrap starting.
I20260807 18:18:22.168092 8436 tablet_bootstrap.cc:654] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa: Neither blocks nor log segments found. Creating new log.
I20260807 18:18:22.169991 8436 log.cc:824] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa: Log is configured to *not* fsync() on all Append() calls
I20260807 18:18:22.171480 8438 tablet_bootstrap.cc:492] T 472bb383ff0c430aa1d54864fe380fe7 P 829552e4c23a43349356c4c34c784c62: Bootstrap starting.
I20260807 18:18:22.172053 8437 tablet_bootstrap.cc:654] T 472bb383ff0c430aa1d54864fe380fe7 P 775ee470852342518a03a533889cc550: Neither blocks nor log segments found. Creating new log.
I20260807 18:18:22.173877 8437 log.cc:824] T 472bb383ff0c430aa1d54864fe380fe7 P 775ee470852342518a03a533889cc550: Log is configured to *not* fsync() on all Append() calls
I20260807 18:18:22.175490 8438 tablet_bootstrap.cc:654] T 472bb383ff0c430aa1d54864fe380fe7 P 829552e4c23a43349356c4c34c784c62: Neither blocks nor log segments found. Creating new log.
I20260807 18:18:22.176946 8438 log.cc:824] T 472bb383ff0c430aa1d54864fe380fe7 P 829552e4c23a43349356c4c34c784c62: Log is configured to *not* fsync() on all Append() calls
I20260807 18:18:22.178081 8436 tablet_bootstrap.cc:492] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa: No bootstrap required, opened a new log
I20260807 18:18:22.178179 8437 tablet_bootstrap.cc:492] T 472bb383ff0c430aa1d54864fe380fe7 P 775ee470852342518a03a533889cc550: No bootstrap required, opened a new log
I20260807 18:18:22.178431 8436 ts_tablet_manager.cc:1397] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.004s
I20260807 18:18:22.178458 8437 ts_tablet_manager.cc:1397] T 472bb383ff0c430aa1d54864fe380fe7 P 775ee470852342518a03a533889cc550: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.006s
I20260807 18:18:22.180615 8438 tablet_bootstrap.cc:492] T 472bb383ff0c430aa1d54864fe380fe7 P 829552e4c23a43349356c4c34c784c62: No bootstrap required, opened a new log
I20260807 18:18:22.181052 8438 ts_tablet_manager.cc:1397] T 472bb383ff0c430aa1d54864fe380fe7 P 829552e4c23a43349356c4c34c784c62: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.005s
I20260807 18:18:22.187003 8436 raft_consensus.cc:348] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "775ee470852342518a03a533889cc550" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38123 } } peers { permanent_uuid: "829552e4c23a43349356c4c34c784c62" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 34987 } } peers { permanent_uuid: "533163cd8b404049ae59946e598492fa" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36227 } }
I20260807 18:18:22.187463 8436 raft_consensus.cc:374] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:18:22.187583 8436 raft_consensus.cc:729] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 533163cd8b404049ae59946e598492fa, State: Initialized, Role: FOLLOWER
I20260807 18:18:22.188289 8436 consensus_queue.cc:260] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "775ee470852342518a03a533889cc550" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38123 } } peers { permanent_uuid: "829552e4c23a43349356c4c34c784c62" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 34987 } } peers { permanent_uuid: "533163cd8b404049ae59946e598492fa" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36227 } }
I20260807 18:18:22.189585 8408 heartbeater.cc:503] Master 127.17.110.62:45375 was elected leader, sending a full tablet report...
I20260807 18:18:22.189744 8437 raft_consensus.cc:348] T 472bb383ff0c430aa1d54864fe380fe7 P 775ee470852342518a03a533889cc550 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "775ee470852342518a03a533889cc550" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38123 } } peers { permanent_uuid: "829552e4c23a43349356c4c34c784c62" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 34987 } } peers { permanent_uuid: "533163cd8b404049ae59946e598492fa" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36227 } }
I20260807 18:18:22.190240 8437 raft_consensus.cc:374] T 472bb383ff0c430aa1d54864fe380fe7 P 775ee470852342518a03a533889cc550 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:18:22.190335 8436 ts_tablet_manager.cc:1428] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa: Time spent starting tablet: real 0.012s user 0.009s sys 0.004s
I20260807 18:18:22.190362 8437 raft_consensus.cc:729] T 472bb383ff0c430aa1d54864fe380fe7 P 775ee470852342518a03a533889cc550 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 775ee470852342518a03a533889cc550, State: Initialized, Role: FOLLOWER
I20260807 18:18:22.191067 8437 consensus_queue.cc:260] T 472bb383ff0c430aa1d54864fe380fe7 P 775ee470852342518a03a533889cc550 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "775ee470852342518a03a533889cc550" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38123 } } peers { permanent_uuid: "829552e4c23a43349356c4c34c784c62" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 34987 } } peers { permanent_uuid: "533163cd8b404049ae59946e598492fa" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36227 } }
I20260807 18:18:22.192389 8214 heartbeater.cc:503] Master 127.17.110.62:45375 was elected leader, sending a full tablet report...
I20260807 18:18:22.193048 8438 raft_consensus.cc:348] T 472bb383ff0c430aa1d54864fe380fe7 P 829552e4c23a43349356c4c34c784c62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "775ee470852342518a03a533889cc550" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38123 } } peers { permanent_uuid: "829552e4c23a43349356c4c34c784c62" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 34987 } } peers { permanent_uuid: "533163cd8b404049ae59946e598492fa" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36227 } }
I20260807 18:18:22.193531 8438 raft_consensus.cc:374] T 472bb383ff0c430aa1d54864fe380fe7 P 829552e4c23a43349356c4c34c784c62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:18:22.193641 8437 ts_tablet_manager.cc:1428] T 472bb383ff0c430aa1d54864fe380fe7 P 775ee470852342518a03a533889cc550: Time spent starting tablet: real 0.015s user 0.010s sys 0.006s
I20260807 18:18:22.193642 8438 raft_consensus.cc:729] T 472bb383ff0c430aa1d54864fe380fe7 P 829552e4c23a43349356c4c34c784c62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 829552e4c23a43349356c4c34c784c62, State: Initialized, Role: FOLLOWER
I20260807 18:18:22.194284 8438 consensus_queue.cc:260] T 472bb383ff0c430aa1d54864fe380fe7 P 829552e4c23a43349356c4c34c784c62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "775ee470852342518a03a533889cc550" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38123 } } peers { permanent_uuid: "829552e4c23a43349356c4c34c784c62" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 34987 } } peers { permanent_uuid: "533163cd8b404049ae59946e598492fa" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36227 } }
I20260807 18:18:22.196276 8438 ts_tablet_manager.cc:1428] T 472bb383ff0c430aa1d54864fe380fe7 P 829552e4c23a43349356c4c34c784c62: Time spent starting tablet: real 0.015s user 0.003s sys 0.010s
W20260807 18:18:22.292454 8410 tablet.cc:2369] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:18:22.322604 8019 tablet.cc:2369] T 472bb383ff0c430aa1d54864fe380fe7 P 829552e4c23a43349356c4c34c784c62: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:18:22.424417 8444 raft_consensus.cc:482] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:18:22.424798 8444 raft_consensus.cc:504] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "775ee470852342518a03a533889cc550" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38123 } } peers { permanent_uuid: "829552e4c23a43349356c4c34c784c62" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 34987 } } peers { permanent_uuid: "533163cd8b404049ae59946e598492fa" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36227 } }
I20260807 18:18:22.426649 8444 leader_election.cc:288] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 775ee470852342518a03a533889cc550 (127.17.110.2:38123), 829552e4c23a43349356c4c34c784c62 (127.17.110.1:34987)
W20260807 18:18:22.433813 8217 tablet.cc:2369] T 472bb383ff0c430aa1d54864fe380fe7 P 775ee470852342518a03a533889cc550: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:18:22.434579 8413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:22.427198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:38123 (local address 127.17.110.3:50511)
0807 18:18:22.427477 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:22.427501 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:22.427739 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:22.428388 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:22.428397 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:22.428426 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:18:22.428741 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:22.428755 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.430323 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.430331 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:22.432131 (+ 1800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:22.432145 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.433635 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.433642 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:22.433944 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:22.433999 (+ 55us) client_negotiation.cc:769] Sending connection context
0807 18:18:22.434106 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 18:18:22.434265 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":49}
I20260807 18:18:22.435231 8429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:22.427417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:50511 (local address 127.17.110.2:38123)
0807 18:18:22.427663 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:22.427669 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:22.427696 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:18:22.427952 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:22.427956 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:22.428108 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:22.428220 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:18:22.428953 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.430161 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.432334 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.433402 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.433654 (+ 252us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:22.433744 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:22.434260 (+ 516us) server_negotiation.cc:299] Negotiation successful
0807 18:18:22.434415 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 18:18:22.436776 8434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:22.427995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:40975 (local address 127.17.110.1:34987)
0807 18:18:22.428270 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:22.428276 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:22.428752 (+ 476us) server_negotiation.cc:407] Connection header received
0807 18:18:22.429196 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:22.429204 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:22.429291 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:22.429358 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:18:22.430259 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.431427 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.434155 (+ 2728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.435534 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.435740 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:22.435822 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:22.436281 (+ 459us) server_negotiation.cc:299] Negotiation successful
0807 18:18:22.436453 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260807 18:18:22.436872 8140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "472bb383ff0c430aa1d54864fe380fe7" candidate_uuid: "533163cd8b404049ae59946e598492fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "775ee470852342518a03a533889cc550" is_pre_election: true
I20260807 18:18:22.437582 8448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:22.427662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:34987 (local address 127.17.110.3:40975)
0807 18:18:22.428625 (+ 963us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:22.428651 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:22.428993 (+ 342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:22.429566 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:22.429570 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:22.429597 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:18:22.430035 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:22.430050 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.431717 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.431724 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:22.433904 (+ 2180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:22.433924 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.435779 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.435786 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:22.436006 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:22.436043 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 18:18:22.436160 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 18:18:22.437174 (+ 1014us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":830,"mutex_wait_us":4,"thread_start_us":747,"threads_started":1}
I20260807 18:18:22.437723 8140 raft_consensus.cc:2454] T 472bb383ff0c430aa1d54864fe380fe7 P 775ee470852342518a03a533889cc550 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 533163cd8b404049ae59946e598492fa in term 0.
I20260807 18:18:22.438670 8247 leader_election.cc:302] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 533163cd8b404049ae59946e598492fa, 775ee470852342518a03a533889cc550; no voters:
I20260807 18:18:22.439160 8444 raft_consensus.cc:2781] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:18:22.438831 7954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "472bb383ff0c430aa1d54864fe380fe7" candidate_uuid: "533163cd8b404049ae59946e598492fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "829552e4c23a43349356c4c34c784c62" is_pre_election: true
I20260807 18:18:22.439296 8444 raft_consensus.cc:482] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:18:22.439463 8444 raft_consensus.cc:3037] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:18:22.439780 7954 raft_consensus.cc:2454] T 472bb383ff0c430aa1d54864fe380fe7 P 829552e4c23a43349356c4c34c784c62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 533163cd8b404049ae59946e598492fa in term 0.
I20260807 18:18:22.442375 8444 raft_consensus.cc:504] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "775ee470852342518a03a533889cc550" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38123 } } peers { permanent_uuid: "829552e4c23a43349356c4c34c784c62" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 34987 } } peers { permanent_uuid: "533163cd8b404049ae59946e598492fa" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36227 } }
I20260807 18:18:22.443360 8444 leader_election.cc:288] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa [CANDIDATE]: Term 1 election: Requested vote from peers 775ee470852342518a03a533889cc550 (127.17.110.2:38123), 829552e4c23a43349356c4c34c784c62 (127.17.110.1:34987)
I20260807 18:18:22.443993 8140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "472bb383ff0c430aa1d54864fe380fe7" candidate_uuid: "533163cd8b404049ae59946e598492fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "775ee470852342518a03a533889cc550"
I20260807 18:18:22.444257 8140 raft_consensus.cc:3037] T 472bb383ff0c430aa1d54864fe380fe7 P 775ee470852342518a03a533889cc550 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:18:22.444370 7954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "472bb383ff0c430aa1d54864fe380fe7" candidate_uuid: "533163cd8b404049ae59946e598492fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "829552e4c23a43349356c4c34c784c62"
I20260807 18:18:22.444554 7954 raft_consensus.cc:3037] T 472bb383ff0c430aa1d54864fe380fe7 P 829552e4c23a43349356c4c34c784c62 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:18:22.447204 7954 raft_consensus.cc:2454] T 472bb383ff0c430aa1d54864fe380fe7 P 829552e4c23a43349356c4c34c784c62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 533163cd8b404049ae59946e598492fa in term 1.
I20260807 18:18:22.447726 8247 leader_election.cc:302] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 533163cd8b404049ae59946e598492fa, 829552e4c23a43349356c4c34c784c62; no voters:
I20260807 18:18:22.447817 8140 raft_consensus.cc:2454] T 472bb383ff0c430aa1d54864fe380fe7 P 775ee470852342518a03a533889cc550 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 533163cd8b404049ae59946e598492fa in term 1.
I20260807 18:18:22.448154 8444 raft_consensus.cc:2781] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:18:22.449365 8444 raft_consensus.cc:686] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa [term 1 LEADER]: Becoming Leader. State: Replica: 533163cd8b404049ae59946e598492fa, State: Running, Role: LEADER
I20260807 18:18:22.450421 8444 consensus_queue.cc:237] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "775ee470852342518a03a533889cc550" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38123 } } peers { permanent_uuid: "829552e4c23a43349356c4c34c784c62" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 34987 } } peers { permanent_uuid: "533163cd8b404049ae59946e598492fa" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36227 } }
I20260807 18:18:22.456640 7743 catalog_manager.cc:5495] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa reported cstate change: term changed from 0 to 1, leader changed from <none> to 533163cd8b404049ae59946e598492fa (127.17.110.3). New cstate: current_term: 1 leader_uuid: "533163cd8b404049ae59946e598492fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "775ee470852342518a03a533889cc550" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "829552e4c23a43349356c4c34c784c62" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 34987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "533163cd8b404049ae59946e598492fa" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36227 } health_report { overall_health: HEALTHY } } }
I20260807 18:18:22.517670 8431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:22.511033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43852 (local address 127.17.110.3:36227)
0807 18:18:22.511267 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:22.511274 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:22.511397 (+ 123us) server_negotiation.cc:407] Connection header received
0807 18:18:22.511723 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:22.511727 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:22.511812 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:22.511995 (+ 183us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:18:22.512432 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.513182 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.515014 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.515536 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.515726 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:22.516351 (+ 625us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:18:22.516945 (+ 594us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:18:22.517138 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:22.517247 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 18:18:22.517345 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 18:18:22.527456 7954 raft_consensus.cc:1261] T 472bb383ff0c430aa1d54864fe380fe7 P 829552e4c23a43349356c4c34c784c62 [term 1 FOLLOWER]: Refusing update from remote peer 533163cd8b404049ae59946e598492fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:18:22.527456 8140 raft_consensus.cc:1261] T 472bb383ff0c430aa1d54864fe380fe7 P 775ee470852342518a03a533889cc550 [term 1 FOLLOWER]: Refusing update from remote peer 533163cd8b404049ae59946e598492fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:18:22.528992 8444 consensus_queue.cc:1035] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "775ee470852342518a03a533889cc550" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:18:22.529491 8449 consensus_queue.cc:1035] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "829552e4c23a43349356c4c34c784c62" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 34987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:18:22.545425 8453 mvcc.cc:205] Tried to move back new op lower bound from 7315974973539581952 to 7315974973243531264. Current Snapshot: MvccSnapshot[applied={T|T < 7315974973539581952}]
I20260807 18:18:22.561823 7743 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37016:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:18:22.562213 7743 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37016:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:18:22.566864 7743 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af: Sending DeleteTablet for 3 replicas of tablet 472bb383ff0c430aa1d54864fe380fe7
I20260807 18:18:22.568415 8313 tablet_service.cc:1514] Processing DeleteTablet for tablet 472bb383ff0c430aa1d54864fe380fe7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:18:22 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:43848
I20260807 18:18:22.568462 8113 tablet_service.cc:1514] Processing DeleteTablet for tablet 472bb383ff0c430aa1d54864fe380fe7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:18:22 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:53080
I20260807 18:18:22.568686 7925 tablet_service.cc:1514] Processing DeleteTablet for tablet 472bb383ff0c430aa1d54864fe380fe7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:18:22 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:52906
I20260807 18:18:22.569460 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 7834
I20260807 18:18:22.578917 8468 tablet_replica.cc:330] T 472bb383ff0c430aa1d54864fe380fe7 P 829552e4c23a43349356c4c34c784c62: stopping tablet replica
I20260807 18:18:22.579332 8466 tablet_replica.cc:330] T 472bb383ff0c430aa1d54864fe380fe7 P 775ee470852342518a03a533889cc550: stopping tablet replica
I20260807 18:18:22.579700 8467 tablet_replica.cc:330] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa: stopping tablet replica
I20260807 18:18:22.579870 8466 raft_consensus.cc:2229] T 472bb383ff0c430aa1d54864fe380fe7 P 775ee470852342518a03a533889cc550 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:18:22.580156 8467 raft_consensus.cc:2229] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:18:22.580518 8466 raft_consensus.cc:2258] T 472bb383ff0c430aa1d54864fe380fe7 P 775ee470852342518a03a533889cc550 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:18:22.580246 8468 raft_consensus.cc:2229] T 472bb383ff0c430aa1d54864fe380fe7 P 829552e4c23a43349356c4c34c784c62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:18:22.580927 8467 raft_consensus.cc:2258] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:18:22.581826 8468 raft_consensus.cc:2258] T 472bb383ff0c430aa1d54864fe380fe7 P 829552e4c23a43349356c4c34c784c62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:18:22.583475 8467 ts_tablet_manager.cc:1905] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:18:22.583472 8466 ts_tablet_manager.cc:1905] T 472bb383ff0c430aa1d54864fe380fe7 P 775ee470852342518a03a533889cc550: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:18:22.584923 8468 ts_tablet_manager.cc:1905] T 472bb383ff0c430aa1d54864fe380fe7 P 829552e4c23a43349356c4c34c784c62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:18:22.591333 8466 ts_tablet_manager.cc:1918] T 472bb383ff0c430aa1d54864fe380fe7 P 775ee470852342518a03a533889cc550: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:18:22.591581 8466 log.cc:1196] T 472bb383ff0c430aa1d54864fe380fe7 P 775ee470852342518a03a533889cc550: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/wal/wals/472bb383ff0c430aa1d54864fe380fe7
I20260807 18:18:22.591739 8467 ts_tablet_manager.cc:1918] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:18:22.591948 8467 log.cc:1196] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/wal/wals/472bb383ff0c430aa1d54864fe380fe7
I20260807 18:18:22.592247 8466 ts_tablet_manager.cc:1939] T 472bb383ff0c430aa1d54864fe380fe7 P 775ee470852342518a03a533889cc550: Deleting consensus metadata
I20260807 18:18:22.592618 8467 ts_tablet_manager.cc:1939] T 472bb383ff0c430aa1d54864fe380fe7 P 533163cd8b404049ae59946e598492fa: Deleting consensus metadata
I20260807 18:18:22.593813 8468 ts_tablet_manager.cc:1918] T 472bb383ff0c430aa1d54864fe380fe7 P 829552e4c23a43349356c4c34c784c62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:18:22.594096 8468 log.cc:1196] T 472bb383ff0c430aa1d54864fe380fe7 P 829552e4c23a43349356c4c34c784c62: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/wal/wals/472bb383ff0c430aa1d54864fe380fe7
I20260807 18:18:22.594571 7725 catalog_manager.cc:4841] TS 533163cd8b404049ae59946e598492fa (127.17.110.3:36227): tablet 472bb383ff0c430aa1d54864fe380fe7 (table test-table [id=af19e6e9c25544e8b070e14794d9576a]) successfully deleted
I20260807 18:18:22.594961 7723 catalog_manager.cc:4841] TS 775ee470852342518a03a533889cc550 (127.17.110.2:38123): tablet 472bb383ff0c430aa1d54864fe380fe7 (table test-table [id=af19e6e9c25544e8b070e14794d9576a]) successfully deleted
I20260807 18:18:22.595671 8468 ts_tablet_manager.cc:1939] T 472bb383ff0c430aa1d54864fe380fe7 P 829552e4c23a43349356c4c34c784c62: Deleting consensus metadata
I20260807 18:18:22.597384 7723 catalog_manager.cc:4841] TS 829552e4c23a43349356c4c34c784c62 (127.17.110.1:34987): tablet 472bb383ff0c430aa1d54864fe380fe7 (table test-table [id=af19e6e9c25544e8b070e14794d9576a]) successfully deleted
Aug 07 18:18:22 dist-test-slave-39l8 krb5kdc[7680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.1@KRBTEST.COM
I20260807 18:18:22.620246 8434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:22.571142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52920 (local address 127.17.110.1:34987)
0807 18:18:22.571391 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:22.571400 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:22.582176 (+ 10776us) server_negotiation.cc:407] Connection header received
0807 18:18:22.582432 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:22.582441 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:22.582572 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:22.582724 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:22.596796 (+ 14072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.597838 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.600291 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.600742 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.600867 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:22.611528 (+ 10661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:22.611562 (+ 34us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:22.611587 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:22.611659 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:22.616002 (+ 4343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:22.617119 (+ 1117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:22.617131 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:22.617135 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:22.617339 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:22.617818 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:22.617824 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:22.617827 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:22.618377 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:22.618832 (+ 455us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:22.619267 (+ 435us) server_negotiation.cc:299] Negotiation successful
0807 18:18:22.619633 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 18:18:22.621411 8010 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:18:22.722590 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 7834
I20260807 18:18:22.752077 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 8025
Aug 07 18:18:22 dist-test-slave-39l8 krb5kdc[7680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.2@KRBTEST.COM
I20260807 18:18:22.775836 8429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:22.753097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53084 (local address 127.17.110.2:38123)
0807 18:18:22.753306 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:22.753313 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:22.753339 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:18:22.753563 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:22.753569 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:22.753676 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:22.753818 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:22.755367 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.756618 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.758541 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.759480 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.759677 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:22.765014 (+ 5337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:22.765066 (+ 52us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:22.765091 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:22.765169 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:22.770115 (+ 4946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:22.771539 (+ 1424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:22.771553 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:22.771558 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:22.771780 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:22.772361 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:22.772367 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:22.772371 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:22.773014 (+ 643us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:22.773385 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:22.774502 (+ 1117us) server_negotiation.cc:299] Negotiation successful
0807 18:18:22.774871 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 18:18:22.777251 8207 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:18:22.877686 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 8025
I20260807 18:18:22.909404 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 8223
Aug 07 18:18:22 dist-test-slave-39l8 krb5kdc[7680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.3@KRBTEST.COM
I20260807 18:18:22.931813 8431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:22.910507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43866 (local address 127.17.110.3:36227)
0807 18:18:22.910824 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:22.910831 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:22.910897 (+ 66us) server_negotiation.cc:407] Connection header received
0807 18:18:22.910973 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:22.910979 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:22.911111 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:22.911241 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:22.912867 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.914072 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.915857 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:22.916657 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:22.916820 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:22.922998 (+ 6178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:22.923021 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:22.923045 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:22.923120 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:22.927830 (+ 4710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:22.928915 (+ 1085us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:22.928935 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:22.928944 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:22.929169 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:22.929672 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:22.929678 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:22.929682 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:22.930265 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:22.930642 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:22.931090 (+ 448us) server_negotiation.cc:299] Negotiation successful
0807 18:18:22.931382 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:18:22.932793 8400 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:18:23.062593 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 8223
I20260807 18:18:23.093619 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 7701
I20260807 18:18:23.095021 7785 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:18:23.212078 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 7701
I20260807 18:18:43.244010 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:45375
--webserver_interface=127.17.110.62
--webserver_port=44325
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:46703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:45375
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:18:43.609690 8476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:18:43.610110 8476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:18:43.610294 8476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:18:43.617967 8476 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 18:18:43.618155 8476 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 18:18:43.618242 8476 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 18:18:43.618345 8476 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:18:43.618431 8476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:18:43.618546 8476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:18:43.618636 8476 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:18:43.618742 8476 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:18:43.629334 8476 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:46703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:45375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:45375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=44325
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:18:43.630883 8476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:18:43.632395 8476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:18:43.661512 8486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:43.661614 8485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:43.661553 8493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:18:43.662442 8476 server_base.cc:1034] running on GCE node
I20260807 18:18:43.663515 8476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:18:43.665395 8476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:18:43.666683 8476 hybrid_clock.cc:648] HybridClock initialized: now 1786126723666585 us; error 45 us; skew 500 ppm
I20260807 18:18:43.667217 8476 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:18:43 dist-test-slave-39l8 krb5kdc[7680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126723, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:18:43.671474 8476 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:18:43.673381 8476 webserver.cc:460] Webserver started at http://127.17.110.62:44325/ using document root <none> and password file <none>
I20260807 18:18:43.674304 8476 fs_manager.cc:359] Metadata directory not provided
I20260807 18:18:43.674431 8476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:18:43.681818 8476 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 18:18:43.685747 8501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:43.688037 8476 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 18:18:43.688344 8476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "7a3b1cd4083644cabeef6647a97bc5af"
format_stamp: "Formatted at 2026-08-07 18:18:20 on dist-test-slave-39l8"
server_key: "c4c13ef066df83212eb28a8223bc959a"
server_key_iv: "ab9f0012e71f5d53b3a51ec2d01a39c1"
server_key_version: "encryptionkey@0"
I20260807 18:18:43.689667 8476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:18:43.729166 8476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:18:43.734930 8476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:18:43.735260 8476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:18:43.762408 8476 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:45375
I20260807 18:18:43.762424 8586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:45375 every 8 connection(s)
I20260807 18:18:43.764334 8476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:18:43.768911 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 8476
I20260807 18:18:43.769945 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.1:34987
--local_ip_for_outbound_sockets=127.17.110.1
--tserver_master_addrs=127.17.110.62:45375
--webserver_port=33745
--webserver_interface=127.17.110.1
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:46703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:18:43.772720 8587 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:18:43.776453 8587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af: Bootstrap starting.
I20260807 18:18:43.803277 8587 log.cc:824] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af: Log is configured to *not* fsync() on all Append() calls
I20260807 18:18:43.817732 8587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:18:43.818359 8587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af: Bootstrap complete.
I20260807 18:18:43.828205 8587 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3b1cd4083644cabeef6647a97bc5af" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45375 } }
I20260807 18:18:43.829144 8587 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a3b1cd4083644cabeef6647a97bc5af, State: Initialized, Role: FOLLOWER
I20260807 18:18:43.829865 8587 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3b1cd4083644cabeef6647a97bc5af" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45375 } }
I20260807 18:18:43.830149 8587 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:18:43.830293 8587 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:18:43.830478 8587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:18:43.834635 8587 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3b1cd4083644cabeef6647a97bc5af" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45375 } }
I20260807 18:18:43.835340 8587 leader_election.cc:302] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a3b1cd4083644cabeef6647a97bc5af; no voters:
I20260807 18:18:43.836050 8587 leader_election.cc:288] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 18:18:43.836205 8592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [term 2 FOLLOWER]: Leader election won for term 2
I20260807 18:18:43.837447 8592 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [term 2 LEADER]: Becoming Leader. State: Replica: 7a3b1cd4083644cabeef6647a97bc5af, State: Running, Role: LEADER
I20260807 18:18:43.838088 8592 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3b1cd4083644cabeef6647a97bc5af" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45375 } }
I20260807 18:18:43.838677 8587 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:18:43.840790 8594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a3b1cd4083644cabeef6647a97bc5af. Latest consensus state: current_term: 2 leader_uuid: "7a3b1cd4083644cabeef6647a97bc5af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3b1cd4083644cabeef6647a97bc5af" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45375 } } }
I20260807 18:18:43.841279 8594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:43.841869 8593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7a3b1cd4083644cabeef6647a97bc5af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3b1cd4083644cabeef6647a97bc5af" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45375 } } }
I20260807 18:18:43.842182 8593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:43.847455 8601 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:18:43.857455 8601 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:18:43.859133 8601 catalog_manager.cc:1259] Loaded cluster ID: ab3bf186d2524cd989e7a485c948352c
I20260807 18:18:43.859283 8601 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:18:43.863662 8601 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:18:43.865528 8601 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af: Loaded TSK: 0
I20260807 18:18:43.896503 8601 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af: Generated new TSK 1
I20260807 18:18:43.896965 8601 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 18:18:44.171890 8589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:18:44.172325 8589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:18:44.172518 8589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:18:44.181443 8589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:18:44.181640 8589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:18:44.181854 8589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.1
I20260807 18:18:44.193789 8589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:46703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.1:34987
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.110.1
--webserver_port=33745
--tserver_master_addrs=127.17.110.62:45375
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.1
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:18:44.195519 8589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:18:44.196998 8589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:18:44.214787 8628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:44.214785 8625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:44.215301 8631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:18:44.216253 8589 server_base.cc:1034] running on GCE node
I20260807 18:18:44.217025 8589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:18:44.218309 8589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:18:44.219594 8589 hybrid_clock.cc:648] HybridClock initialized: now 1786126724219530 us; error 48 us; skew 500 ppm
I20260807 18:18:44.219973 8589 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:18:44 dist-test-slave-39l8 krb5kdc[7680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126724, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:18:44.225634 8589 init.cc:377] Logged in from keytab as kudu/127.17.110.1@KRBTEST.COM (short username kudu)
I20260807 18:18:44.228296 8589 webserver.cc:460] Webserver started at http://127.17.110.1:33745/ using document root <none> and password file <none>
I20260807 18:18:44.229178 8589 fs_manager.cc:359] Metadata directory not provided
I20260807 18:18:44.229384 8589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:18:44.236308 8589 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 18:18:44.240028 8641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:44.241931 8589 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:18:44.242206 8589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/wal
uuid: "829552e4c23a43349356c4c34c784c62"
format_stamp: "Formatted at 2026-08-07 18:18:20 on dist-test-slave-39l8"
server_key: "dcc5621c19c74d9c7e41f6c9fd89a00f"
server_key_iv: "29553daaa980ebbd0043c8251d86ee53"
server_key_version: "encryptionkey@0"
I20260807 18:18:44.243475 8589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:18:44.261090 8589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:18:44.266449 8589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:18:44.266865 8589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:18:44.268121 8589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:18:44.269850 8589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:18:44.270004 8589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:44.270154 8589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:18:44.270252 8589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:44.308159 8589 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.1:34987
I20260807 18:18:44.308269 8812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.1:34987 every 8 connection(s)
I20260807 18:18:44.309882 8589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
I20260807 18:18:44.317194 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 8589
I20260807 18:18:44.318169 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.2:38123
--local_ip_for_outbound_sockets=127.17.110.2
--tserver_master_addrs=127.17.110.62:45375
--webserver_port=43341
--webserver_interface=127.17.110.2
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:46703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:18:44 dist-test-slave-39l8 krb5kdc[7680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126724, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:18:44.340569 8820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:44.313300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.1:58593)
0807 18:18:44.314191 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:44.314233 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:44.316112 (+ 1879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:44.318237 (+ 2125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:44.318245 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:44.319023 (+ 778us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:18:44.319688 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:44.319710 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:44.322620 (+ 2910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:44.322625 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:44.324009 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:44.324027 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:44.325784 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:44.325789 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:44.326008 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:44.326883 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:18:44.326910 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:18:44.329789 (+ 2879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:18:44.335406 (+ 5617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:44.335416 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:44.335431 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:44.335978 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:44.336657 (+ 679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:44.336662 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:44.336664 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:44.336898 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:44.338000 (+ 1102us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:18:44.338006 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:18:44.338379 (+ 373us) client_negotiation.cc:769] Sending connection context
0807 18:18:44.338654 (+ 275us) client_negotiation.cc:240] Negotiation successful
0807 18:18:44.339320 (+ 666us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":355,"threads_started":1}
I20260807 18:18:44.340819 8816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:44.313142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:58593 (local address 127.17.110.62:45375)
0807 18:18:44.313882 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:44.313900 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:44.314395 (+ 495us) server_negotiation.cc:407] Connection header received
0807 18:18:44.316951 (+ 2556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:44.316975 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:44.317595 (+ 620us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:44.318102 (+ 507us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:44.319990 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:44.322326 (+ 2336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:44.324206 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:44.325488 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:44.325762 (+ 274us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:44.330206 (+ 4444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:44.330215 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:44.330234 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:44.330277 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:44.335134 (+ 4857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:44.336183 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:44.336198 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:44.336209 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:44.336483 (+ 274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:44.337054 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:44.337062 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:44.337068 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:44.337824 (+ 756us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:44.338051 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:44.338968 (+ 917us) server_negotiation.cc:299] Negotiation successful
0807 18:18:44.339497 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":281,"threads_started":1}
I20260807 18:18:44.344273 8814 heartbeater.cc:347] Connected to a master server at 127.17.110.62:45375
I20260807 18:18:44.344667 8814 heartbeater.cc:464] Registering TS with master...
I20260807 18:18:44.345688 8814 heartbeater.cc:511] Master 127.17.110.62:45375 requested a full tablet report, sending...
I20260807 18:18:44.348973 8530 ts_manager.cc:194] Registered new tserver with Master: 829552e4c23a43349356c4c34c784c62 (127.17.110.1:34987)
I20260807 18:18:44.351365 8530 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.1@KRBTEST.COM'} at 127.17.110.1:58593
W20260807 18:18:44.687841 8821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:18:44.688169 8821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:18:44.688331 8821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:18:44.696261 8821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:18:44.696390 8821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:18:44.696600 8821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.2
I20260807 18:18:44.707069 8821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:46703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.2:38123
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.110.2
--webserver_port=43341
--tserver_master_addrs=127.17.110.62:45375
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.2
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:18:44.708531 8821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:18:44.709908 8821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:18:44.725116 8830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:44.725160 8834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:44.725212 8831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:18:44.726127 8821 server_base.cc:1034] running on GCE node
I20260807 18:18:44.726922 8821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:18:44.727975 8821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:18:44.729216 8821 hybrid_clock.cc:648] HybridClock initialized: now 1786126724729181 us; error 35 us; skew 500 ppm
I20260807 18:18:44.729579 8821 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:18:44 dist-test-slave-39l8 krb5kdc[7680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126724, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:18:44.733471 8821 init.cc:377] Logged in from keytab as kudu/127.17.110.2@KRBTEST.COM (short username kudu)
I20260807 18:18:44.735204 8821 webserver.cc:460] Webserver started at http://127.17.110.2:43341/ using document root <none> and password file <none>
I20260807 18:18:44.735991 8821 fs_manager.cc:359] Metadata directory not provided
I20260807 18:18:44.736166 8821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:18:44.742314 8821 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:18:44.745491 8844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:44.747124 8821 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 18:18:44.747363 8821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/wal
uuid: "775ee470852342518a03a533889cc550"
format_stamp: "Formatted at 2026-08-07 18:18:21 on dist-test-slave-39l8"
server_key: "345d8a4e5013cf01743d32339088b2d4"
server_key_iv: "66e985761c60acea9673dba49f81f3ff"
server_key_version: "encryptionkey@0"
I20260807 18:18:44.748576 8821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:18:44.766396 8821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:18:44.771435 8821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:18:44.771752 8821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:18:44.772994 8821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:18:44.774473 8821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:18:44.774609 8821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:44.774763 8821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:18:44.774864 8821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260807 18:18:44.811501 8821 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.2:38123
I20260807 18:18:44.811638 9022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.2:38123 every 8 connection(s)
I20260807 18:18:44.813129 8821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
I20260807 18:18:44.816512 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 8821
I20260807 18:18:44.817416 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.3:36227
--local_ip_for_outbound_sockets=127.17.110.3
--tserver_master_addrs=127.17.110.62:45375
--webserver_port=43873
--webserver_interface=127.17.110.3
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:46703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:18:44 dist-test-slave-39l8 krb5kdc[7680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126724, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:18:44.841038 8816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:44.816071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:51463 (local address 127.17.110.62:45375)
0807 18:18:44.816351 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:44.816358 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:44.817857 (+ 1499us) server_negotiation.cc:407] Connection header received
0807 18:18:44.820993 (+ 3136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:44.820998 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:44.821170 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:44.821271 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:44.823719 (+ 2448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:44.824487 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:44.827203 (+ 2716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:44.827685 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:44.827799 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:44.832709 (+ 4910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:44.832719 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:44.832727 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:44.832781 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:44.836794 (+ 4013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:44.837838 (+ 1044us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:44.837848 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:44.837854 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:44.838065 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:44.838585 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:44.838593 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:44.838598 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:44.839087 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:44.839265 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:44.840247 (+ 982us) server_negotiation.cc:299] Negotiation successful
0807 18:18:44.840568 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 18:18:44.841826 9025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:44.816545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.2:51463)
0807 18:18:44.817498 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:44.817691 (+ 193us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:44.820603 (+ 2912us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:44.821604 (+ 1001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:44.821618 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:44.822490 (+ 872us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:18:44.823463 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:44.823496 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:44.824681 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:44.824690 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:44.827010 (+ 2320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:44.827031 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:44.827960 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:44.827970 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:44.828332 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:44.829242 (+ 910us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:18:44.829281 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:18:44.832348 (+ 3067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:18:44.837047 (+ 4699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:44.837057 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:44.837075 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:44.837644 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:44.838213 (+ 569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:44.838217 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:44.838220 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:44.838444 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:44.839293 (+ 849us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:18:44.839301 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:18:44.839730 (+ 429us) client_negotiation.cc:769] Sending connection context
0807 18:18:44.840078 (+ 348us) client_negotiation.cc:240] Negotiation successful
0807 18:18:44.840609 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":350,"threads_started":1}
I20260807 18:18:44.843611 9023 heartbeater.cc:347] Connected to a master server at 127.17.110.62:45375
I20260807 18:18:44.843967 9023 heartbeater.cc:464] Registering TS with master...
I20260807 18:18:44.844760 9023 heartbeater.cc:511] Master 127.17.110.62:45375 requested a full tablet report, sending...
I20260807 18:18:44.846688 8530 ts_manager.cc:194] Registered new tserver with Master: 775ee470852342518a03a533889cc550 (127.17.110.2:38123)
I20260807 18:18:44.847805 8530 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.2@KRBTEST.COM'} at 127.17.110.2:51463
W20260807 18:18:45.187150 9026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:18:45.187531 9026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:18:45.187716 9026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:18:45.195542 9026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:18:45.195735 9026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:18:45.195953 9026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.3
I20260807 18:18:45.206518 9026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:46703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.3:36227
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.110.3
--webserver_port=43873
--tserver_master_addrs=127.17.110.62:45375
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.3
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:18:45.208021 9026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:18:45.209611 9026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:18:45.225443 9042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:45.225497 9039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:45.226541 9044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:18:45.227881 9026 server_base.cc:1034] running on GCE node
I20260807 18:18:45.229748 9026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:18:45.231316 9026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:18:45.232648 9026 hybrid_clock.cc:648] HybridClock initialized: now 1786126725232607 us; error 64 us; skew 500 ppm
I20260807 18:18:45.233139 9026 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:18:45 dist-test-slave-39l8 krb5kdc[7680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126725, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:18:45.237711 9026 init.cc:377] Logged in from keytab as kudu/127.17.110.3@KRBTEST.COM (short username kudu)
I20260807 18:18:45.240291 9026 webserver.cc:460] Webserver started at http://127.17.110.3:43873/ using document root <none> and password file <none>
I20260807 18:18:45.241439 9026 fs_manager.cc:359] Metadata directory not provided
I20260807 18:18:45.241627 9026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:18:45.249706 9026 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 18:18:45.253882 9056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:45.256024 9026 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:18:45.256291 9026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/wal
uuid: "533163cd8b404049ae59946e598492fa"
format_stamp: "Formatted at 2026-08-07 18:18:21 on dist-test-slave-39l8"
server_key: "550f862ace80c150582d7ab6bead54f5"
server_key_iv: "ce733b93103111465bb94eabed1ccfd6"
server_key_version: "encryptionkey@0"
I20260807 18:18:45.257493 9026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:18:45.283587 9026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:18:45.289893 9026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:18:45.290302 9026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:18:45.292014 9026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:18:45.293761 9026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:18:45.293877 9026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:45.294028 9026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:18:45.294108 9026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:45.333956 9026 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.3:36227
I20260807 18:18:45.334060 9235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.3:36227 every 8 connection(s)
I20260807 18:18:45.335711 9026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
I20260807 18:18:45.337136 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 9026
Aug 07 18:18:45 dist-test-slave-39l8 krb5kdc[7680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126725, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:18:45.356120 8814 heartbeater.cc:503] Master 127.17.110.62:45375 was elected leader, sending a full tablet report...
I20260807 18:18:45.357854 8816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:45.338452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53728 (local address 127.17.110.62:45375)
0807 18:18:45.338674 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:45.338680 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:45.339776 (+ 1096us) server_negotiation.cc:407] Connection header received
0807 18:18:45.339932 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:45.339940 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:45.340049 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:45.340186 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:45.342036 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.343220 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.345672 (+ 2452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.346266 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.346388 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:45.350603 (+ 4215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:45.350610 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:45.350613 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:45.350660 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:45.354301 (+ 3641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:45.355222 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:45.355238 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:45.355242 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:45.355383 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:45.355900 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:45.355904 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:45.355906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:45.356196 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:45.356490 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:45.357179 (+ 689us) server_negotiation.cc:299] Negotiation successful
0807 18:18:45.357438 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 18:18:45.366650 9242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:45.338732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:58549 (local address 127.17.110.62:45375)
0807 18:18:45.339303 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:45.339309 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:45.342781 (+ 3472us) server_negotiation.cc:407] Connection header received
0807 18:18:45.345800 (+ 3019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:45.345810 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:45.345930 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:45.346078 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:45.348640 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.349496 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.351515 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.352421 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.352601 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:45.356947 (+ 4346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:45.356957 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:45.356964 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:45.357007 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:45.361351 (+ 4344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:45.362623 (+ 1272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:45.362629 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:45.362633 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:45.363044 (+ 411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:45.364032 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:45.364037 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:45.364039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:45.364735 (+ 696us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:45.364895 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:45.365909 (+ 1014us) server_negotiation.cc:299] Negotiation successful
0807 18:18:45.366190 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":285,"threads_started":1}
I20260807 18:18:45.367785 9243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:45.339617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.3:58549)
0807 18:18:45.342536 (+ 2919us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:45.342628 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:45.345365 (+ 2737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:45.346450 (+ 1085us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:45.346464 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:45.347612 (+ 1148us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:18:45.348367 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:45.348393 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.349713 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.349719 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:45.351256 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:45.351274 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.352805 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.352817 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:45.353211 (+ 394us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:45.353986 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:18:45.354014 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:18:45.356606 (+ 2592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:18:45.361603 (+ 4997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:45.361616 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:45.361637 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:45.362379 (+ 742us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:45.363285 (+ 906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:45.363301 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:45.363304 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:45.363846 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:45.364976 (+ 1130us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:18:45.364984 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:18:45.365468 (+ 484us) client_negotiation.cc:769] Sending connection context
0807 18:18:45.365756 (+ 288us) client_negotiation.cc:240] Negotiation successful
0807 18:18:45.366241 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2519,"thread_start_us":2288,"threads_started":1}
I20260807 18:18:45.370333 9236 heartbeater.cc:347] Connected to a master server at 127.17.110.62:45375
I20260807 18:18:45.370792 9236 heartbeater.cc:464] Registering TS with master...
I20260807 18:18:45.371697 9236 heartbeater.cc:511] Master 127.17.110.62:45375 requested a full tablet report, sending...
I20260807 18:18:45.373903 8530 ts_manager.cc:194] Registered new tserver with Master: 533163cd8b404049ae59946e598492fa (127.17.110.3:36227)
I20260807 18:18:45.375037 8530 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.3@KRBTEST.COM'} at 127.17.110.3:58549
I20260807 18:18:45.375716 17848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:18:45.384177 9242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:45.377006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53738 (local address 127.17.110.62:45375)
0807 18:18:45.377277 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:45.377288 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:45.377849 (+ 561us) server_negotiation.cc:407] Connection header received
0807 18:18:45.378083 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:45.378087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:45.378287 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:45.378439 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:18:45.379236 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.380092 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.381878 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.382351 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.382467 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:45.383252 (+ 785us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:18:45.383502 (+ 250us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 18:18:45.383867 (+ 365us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53738: token expired
Metrics: {"server-negotiator.queue_time_us":76}
W20260807 18:18:45.384342 9242 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53738: token expired
W20260807 18:18:45.384974 8417 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.17.110.62:45375)
I20260807 18:18:45.401012 9242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:45.385682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53746 (local address 127.17.110.62:45375)
0807 18:18:45.385993 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:45.385998 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:45.386017 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:18:45.386068 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:45.386072 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:45.386200 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:45.386285 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:45.388126 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.389030 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.390569 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.391141 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.391261 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:45.394784 (+ 3523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:45.394807 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:45.394811 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:45.394857 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:45.397765 (+ 2908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:45.398661 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:45.398666 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:45.398668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:45.398925 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:45.399473 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:45.399477 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:45.399479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:45.399848 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:45.400072 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:45.400477 (+ 405us) server_negotiation.cc:299] Negotiation successful
0807 18:18:45.400666 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260807 18:18:45.405817 8530 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53746:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:18:45.417428 8530 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:18:45.461500 9253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:45.435153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:34987 (local address 127.0.0.1:38182)
0807 18:18:45.451112 (+ 15959us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:45.451362 (+ 250us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:45.451672 (+ 310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:45.453085 (+ 1413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:45.453098 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:45.453152 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:18:45.453943 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:45.453967 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.456272 (+ 2305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.456279 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:45.458355 (+ 2076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:45.458376 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.460113 (+ 1737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.460121 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:45.460444 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:45.460543 (+ 99us) client_negotiation.cc:769] Sending connection context
0807 18:18:45.460741 (+ 198us) client_negotiation.cc:240] Negotiation successful
0807 18:18:45.461002 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15481,"thread_start_us":15210,"threads_started":1}
I20260807 18:18:45.462212 9254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:45.435396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38182 (local address 127.17.110.1:34987)
0807 18:18:45.450201 (+ 14805us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:45.450213 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:45.451984 (+ 1771us) server_negotiation.cc:407] Connection header received
0807 18:18:45.452232 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:45.452247 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:45.452698 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:45.454246 (+ 1548us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:18:45.454558 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.456091 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.458601 (+ 2510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.459793 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.459998 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:45.460308 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:45.460948 (+ 640us) server_negotiation.cc:299] Negotiation successful
0807 18:18:45.461214 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14490,"thread_start_us":14356,"threads_started":1}
I20260807 18:18:45.467617 9255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:45.435727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:36227 (local address 127.0.0.1:36430)
0807 18:18:45.454578 (+ 18851us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:45.454605 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:45.454774 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:45.457132 (+ 2358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:45.457141 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:45.457199 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:18:45.457866 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:45.457890 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.461189 (+ 3299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.461197 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:45.465052 (+ 3855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:45.465069 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.466572 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.466580 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:45.466829 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:45.466866 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 18:18:45.467020 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 18:18:45.467179 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18625,"thread_start_us":18374,"threads_started":1}
I20260807 18:18:45.467648 9256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:45.436300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36430 (local address 127.17.110.3:36227)
0807 18:18:45.455517 (+ 19217us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:45.455529 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:45.455593 (+ 64us) server_negotiation.cc:407] Connection header received
0807 18:18:45.455751 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:45.455765 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:45.456137 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:45.456446 (+ 309us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:18:45.458235 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.460865 (+ 2630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.465269 (+ 4404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.466294 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.466483 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:45.466799 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:45.467074 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 18:18:45.467255 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18943,"thread_start_us":20506,"threads_started":1}
I20260807 18:18:45.467324 8718 tablet_service.cc:1467] Processing CreateTablet for tablet 3ab00d0f3a564df5aa430bfa063b25aa (DEFAULT_TABLE table=test-table [id=a12321c9933b485e97194967cecabdc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:18:45.469439 8718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ab00d0f3a564df5aa430bfa063b25aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:18:45.470829 9259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:45.455005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:38123 (local address 127.0.0.1:33954)
0807 18:18:45.460905 (+ 5900us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:45.460927 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:45.461076 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:45.462439 (+ 1363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:45.462444 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:45.462474 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:18:45.462914 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:45.462936 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.465009 (+ 2073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.465017 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:45.467497 (+ 2480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:45.467520 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.469861 (+ 2341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.469870 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:45.470102 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:45.470141 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 18:18:45.470260 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 18:18:45.470414 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5744,"thread_start_us":5523,"threads_started":1}
I20260807 18:18:45.471731 9257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:45.452029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33954 (local address 127.17.110.2:38123)
0807 18:18:45.454865 (+ 2836us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:45.454877 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:45.461255 (+ 6378us) server_negotiation.cc:407] Connection header received
0807 18:18:45.461389 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:45.461398 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:45.461754 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:45.462004 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:18:45.463120 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.464802 (+ 1682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.467783 (+ 2981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.469557 (+ 1774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.469723 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:45.469948 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:45.471210 (+ 1262us) server_negotiation.cc:299] Negotiation successful
0807 18:18:45.471429 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2620,"thread_start_us":2475,"threads_started":1}
I20260807 18:18:45.472177 9132 tablet_service.cc:1467] Processing CreateTablet for tablet 3ab00d0f3a564df5aa430bfa063b25aa (DEFAULT_TABLE table=test-table [id=a12321c9933b485e97194967cecabdc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:18:45.473749 9132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ab00d0f3a564df5aa430bfa063b25aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:18:45.476346 8916 tablet_service.cc:1467] Processing CreateTablet for tablet 3ab00d0f3a564df5aa430bfa063b25aa (DEFAULT_TABLE table=test-table [id=a12321c9933b485e97194967cecabdc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:18:45.478031 8916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ab00d0f3a564df5aa430bfa063b25aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:18:45.485761 9260 tablet_bootstrap.cc:492] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62: Bootstrap starting.
I20260807 18:18:45.489739 9260 tablet_bootstrap.cc:654] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62: Neither blocks nor log segments found. Creating new log.
I20260807 18:18:45.491241 9261 tablet_bootstrap.cc:492] T 3ab00d0f3a564df5aa430bfa063b25aa P 533163cd8b404049ae59946e598492fa: Bootstrap starting.
I20260807 18:18:45.493806 9260 log.cc:824] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62: Log is configured to *not* fsync() on all Append() calls
I20260807 18:18:45.496694 9261 tablet_bootstrap.cc:654] T 3ab00d0f3a564df5aa430bfa063b25aa P 533163cd8b404049ae59946e598492fa: Neither blocks nor log segments found. Creating new log.
I20260807 18:18:45.498807 9261 log.cc:824] T 3ab00d0f3a564df5aa430bfa063b25aa P 533163cd8b404049ae59946e598492fa: Log is configured to *not* fsync() on all Append() calls
I20260807 18:18:45.501102 9264 tablet_bootstrap.cc:492] T 3ab00d0f3a564df5aa430bfa063b25aa P 775ee470852342518a03a533889cc550: Bootstrap starting.
I20260807 18:18:45.503052 9260 tablet_bootstrap.cc:492] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62: No bootstrap required, opened a new log
I20260807 18:18:45.503432 9260 ts_tablet_manager.cc:1397] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.007s
I20260807 18:18:45.503611 9261 tablet_bootstrap.cc:492] T 3ab00d0f3a564df5aa430bfa063b25aa P 533163cd8b404049ae59946e598492fa: No bootstrap required, opened a new log
I20260807 18:18:45.503952 9261 ts_tablet_manager.cc:1397] T 3ab00d0f3a564df5aa430bfa063b25aa P 533163cd8b404049ae59946e598492fa: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.004s
I20260807 18:18:45.505250 9264 tablet_bootstrap.cc:654] T 3ab00d0f3a564df5aa430bfa063b25aa P 775ee470852342518a03a533889cc550: Neither blocks nor log segments found. Creating new log.
I20260807 18:18:45.506762 9264 log.cc:824] T 3ab00d0f3a564df5aa430bfa063b25aa P 775ee470852342518a03a533889cc550: Log is configured to *not* fsync() on all Append() calls
I20260807 18:18:45.510561 9264 tablet_bootstrap.cc:492] T 3ab00d0f3a564df5aa430bfa063b25aa P 775ee470852342518a03a533889cc550: No bootstrap required, opened a new log
I20260807 18:18:45.510975 9264 ts_tablet_manager.cc:1397] T 3ab00d0f3a564df5aa430bfa063b25aa P 775ee470852342518a03a533889cc550: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.007s
I20260807 18:18:45.512773 9261 raft_consensus.cc:348] T 3ab00d0f3a564df5aa430bfa063b25aa P 533163cd8b404049ae59946e598492fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829552e4c23a43349356c4c34c784c62" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 34987 } } peers { permanent_uuid: "775ee470852342518a03a533889cc550" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38123 } } peers { permanent_uuid: "533163cd8b404049ae59946e598492fa" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36227 } }
I20260807 18:18:45.513214 9261 raft_consensus.cc:374] T 3ab00d0f3a564df5aa430bfa063b25aa P 533163cd8b404049ae59946e598492fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:18:45.513319 9261 raft_consensus.cc:729] T 3ab00d0f3a564df5aa430bfa063b25aa P 533163cd8b404049ae59946e598492fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 533163cd8b404049ae59946e598492fa, State: Initialized, Role: FOLLOWER
I20260807 18:18:45.514015 9261 consensus_queue.cc:260] T 3ab00d0f3a564df5aa430bfa063b25aa P 533163cd8b404049ae59946e598492fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829552e4c23a43349356c4c34c784c62" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 34987 } } peers { permanent_uuid: "775ee470852342518a03a533889cc550" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38123 } } peers { permanent_uuid: "533163cd8b404049ae59946e598492fa" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36227 } }
I20260807 18:18:45.515317 9236 heartbeater.cc:503] Master 127.17.110.62:45375 was elected leader, sending a full tablet report...
I20260807 18:18:45.514976 9260 raft_consensus.cc:348] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829552e4c23a43349356c4c34c784c62" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 34987 } } peers { permanent_uuid: "775ee470852342518a03a533889cc550" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38123 } } peers { permanent_uuid: "533163cd8b404049ae59946e598492fa" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36227 } }
I20260807 18:18:45.515496 9260 raft_consensus.cc:374] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:18:45.515591 9260 raft_consensus.cc:729] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 829552e4c23a43349356c4c34c784c62, State: Initialized, Role: FOLLOWER
I20260807 18:18:45.516089 9261 ts_tablet_manager.cc:1428] T 3ab00d0f3a564df5aa430bfa063b25aa P 533163cd8b404049ae59946e598492fa: Time spent starting tablet: real 0.012s user 0.007s sys 0.005s
I20260807 18:18:45.516287 9260 consensus_queue.cc:260] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829552e4c23a43349356c4c34c784c62" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 34987 } } peers { permanent_uuid: "775ee470852342518a03a533889cc550" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38123 } } peers { permanent_uuid: "533163cd8b404049ae59946e598492fa" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36227 } }
I20260807 18:18:45.520574 9260 ts_tablet_manager.cc:1428] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62: Time spent starting tablet: real 0.017s user 0.013s sys 0.001s
I20260807 18:18:45.522879 9264 raft_consensus.cc:348] T 3ab00d0f3a564df5aa430bfa063b25aa P 775ee470852342518a03a533889cc550 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829552e4c23a43349356c4c34c784c62" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 34987 } } peers { permanent_uuid: "775ee470852342518a03a533889cc550" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38123 } } peers { permanent_uuid: "533163cd8b404049ae59946e598492fa" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36227 } }
I20260807 18:18:45.523530 9264 raft_consensus.cc:374] T 3ab00d0f3a564df5aa430bfa063b25aa P 775ee470852342518a03a533889cc550 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:18:45.523690 9264 raft_consensus.cc:729] T 3ab00d0f3a564df5aa430bfa063b25aa P 775ee470852342518a03a533889cc550 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 775ee470852342518a03a533889cc550, State: Initialized, Role: FOLLOWER
I20260807 18:18:45.524461 9264 consensus_queue.cc:260] T 3ab00d0f3a564df5aa430bfa063b25aa P 775ee470852342518a03a533889cc550 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829552e4c23a43349356c4c34c784c62" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 34987 } } peers { permanent_uuid: "775ee470852342518a03a533889cc550" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38123 } } peers { permanent_uuid: "533163cd8b404049ae59946e598492fa" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36227 } }
I20260807 18:18:45.525853 9023 heartbeater.cc:503] Master 127.17.110.62:45375 was elected leader, sending a full tablet report...
I20260807 18:18:45.526623 9264 ts_tablet_manager.cc:1428] T 3ab00d0f3a564df5aa430bfa063b25aa P 775ee470852342518a03a533889cc550: Time spent starting tablet: real 0.016s user 0.013s sys 0.001s
W20260807 18:18:45.562651 8815 tablet.cc:2369] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:18:45.565351 9024 tablet.cc:2369] T 3ab00d0f3a564df5aa430bfa063b25aa P 775ee470852342518a03a533889cc550: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:18:45.587584 9240 tablet.cc:2369] T 3ab00d0f3a564df5aa430bfa063b25aa P 533163cd8b404049ae59946e598492fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:18:45.711382 9270 raft_consensus.cc:482] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:18:45.711790 9270 raft_consensus.cc:504] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829552e4c23a43349356c4c34c784c62" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 34987 } } peers { permanent_uuid: "775ee470852342518a03a533889cc550" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38123 } } peers { permanent_uuid: "533163cd8b404049ae59946e598492fa" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36227 } }
I20260807 18:18:45.713775 9270 leader_election.cc:288] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 775ee470852342518a03a533889cc550 (127.17.110.2:38123), 533163cd8b404049ae59946e598492fa (127.17.110.3:36227)
I20260807 18:18:45.721928 9257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:45.714351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:43307 (local address 127.17.110.2:38123)
0807 18:18:45.714600 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:45.714605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:45.715272 (+ 667us) server_negotiation.cc:407] Connection header received
0807 18:18:45.715514 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:45.715520 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:45.715610 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:45.715708 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:18:45.716473 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.717336 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.719700 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.720586 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.720775 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:45.721293 (+ 518us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:45.721465 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 18:18:45.721623 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 18:18:45.721969 9256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:45.714268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:50101 (local address 127.17.110.3:36227)
0807 18:18:45.714499 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:45.714506 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:45.715508 (+ 1002us) server_negotiation.cc:407] Connection header received
0807 18:18:45.715717 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:45.715722 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:45.715813 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:45.715928 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:18:45.716611 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.717439 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.719735 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.720610 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.720841 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:45.720929 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:45.721542 (+ 613us) server_negotiation.cc:299] Negotiation successful
0807 18:18:45.721700 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 18:18:45.722078 9273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:45.714196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:38123 (local address 127.17.110.1:43307)
0807 18:18:45.715100 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:45.715127 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:45.715337 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:45.715748 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:45.715753 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:45.715779 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:18:45.716287 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:45.716302 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.717519 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.717527 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:45.719451 (+ 1924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:45.719472 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.720872 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.720880 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:45.721265 (+ 385us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:45.721304 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 18:18:45.721404 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 18:18:45.721593 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"mutex_wait_us":236,"thread_start_us":439,"threads_started":1}
I20260807 18:18:45.722600 9275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:45.714396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:36227 (local address 127.17.110.1:50101)
0807 18:18:45.715332 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:45.715355 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:45.715521 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:45.715965 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:45.715968 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:45.715989 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:18:45.716415 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:45.716431 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.717613 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.717617 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:45.719451 (+ 1834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:45.719472 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.720932 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.720941 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:45.721290 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:45.721335 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 18:18:45.721470 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 18:18:45.721633 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":766,"mutex_wait_us":76,"thread_start_us":682,"threads_started":1}
I20260807 18:18:45.723469 8949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ab00d0f3a564df5aa430bfa063b25aa" candidate_uuid: "829552e4c23a43349356c4c34c784c62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "775ee470852342518a03a533889cc550" is_pre_election: true
I20260807 18:18:45.723730 9173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ab00d0f3a564df5aa430bfa063b25aa" candidate_uuid: "829552e4c23a43349356c4c34c784c62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "533163cd8b404049ae59946e598492fa" is_pre_election: true
I20260807 18:18:45.724146 8949 raft_consensus.cc:2454] T 3ab00d0f3a564df5aa430bfa063b25aa P 775ee470852342518a03a533889cc550 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 829552e4c23a43349356c4c34c784c62 in term 0.
I20260807 18:18:45.724273 9173 raft_consensus.cc:2454] T 3ab00d0f3a564df5aa430bfa063b25aa P 533163cd8b404049ae59946e598492fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 829552e4c23a43349356c4c34c784c62 in term 0.
I20260807 18:18:45.725203 8649 leader_election.cc:302] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 533163cd8b404049ae59946e598492fa, 829552e4c23a43349356c4c34c784c62; no voters:
I20260807 18:18:45.725772 9270 raft_consensus.cc:2781] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:18:45.725950 9270 raft_consensus.cc:482] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:18:45.726063 9270 raft_consensus.cc:3037] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:18:45.728629 9270 raft_consensus.cc:504] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829552e4c23a43349356c4c34c784c62" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 34987 } } peers { permanent_uuid: "775ee470852342518a03a533889cc550" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38123 } } peers { permanent_uuid: "533163cd8b404049ae59946e598492fa" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36227 } }
I20260807 18:18:45.729699 9270 leader_election.cc:288] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62 [CANDIDATE]: Term 1 election: Requested vote from peers 775ee470852342518a03a533889cc550 (127.17.110.2:38123), 533163cd8b404049ae59946e598492fa (127.17.110.3:36227)
I20260807 18:18:45.730063 8949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ab00d0f3a564df5aa430bfa063b25aa" candidate_uuid: "829552e4c23a43349356c4c34c784c62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "775ee470852342518a03a533889cc550"
I20260807 18:18:45.730175 9173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ab00d0f3a564df5aa430bfa063b25aa" candidate_uuid: "829552e4c23a43349356c4c34c784c62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "533163cd8b404049ae59946e598492fa"
I20260807 18:18:45.730324 8949 raft_consensus.cc:3037] T 3ab00d0f3a564df5aa430bfa063b25aa P 775ee470852342518a03a533889cc550 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:18:45.730460 9173 raft_consensus.cc:3037] T 3ab00d0f3a564df5aa430bfa063b25aa P 533163cd8b404049ae59946e598492fa [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:18:45.733053 8949 raft_consensus.cc:2454] T 3ab00d0f3a564df5aa430bfa063b25aa P 775ee470852342518a03a533889cc550 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 829552e4c23a43349356c4c34c784c62 in term 1.
I20260807 18:18:45.733078 9173 raft_consensus.cc:2454] T 3ab00d0f3a564df5aa430bfa063b25aa P 533163cd8b404049ae59946e598492fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 829552e4c23a43349356c4c34c784c62 in term 1.
I20260807 18:18:45.733734 8643 leader_election.cc:302] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 775ee470852342518a03a533889cc550, 829552e4c23a43349356c4c34c784c62; no voters:
I20260807 18:18:45.734105 9270 raft_consensus.cc:2781] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:18:45.734673 9270 raft_consensus.cc:686] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62 [term 1 LEADER]: Becoming Leader. State: Replica: 829552e4c23a43349356c4c34c784c62, State: Running, Role: LEADER
I20260807 18:18:45.735268 9270 consensus_queue.cc:237] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829552e4c23a43349356c4c34c784c62" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 34987 } } peers { permanent_uuid: "775ee470852342518a03a533889cc550" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38123 } } peers { permanent_uuid: "533163cd8b404049ae59946e598492fa" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36227 } }
I20260807 18:18:45.740397 8526 catalog_manager.cc:5495] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62 reported cstate change: term changed from 0 to 1, leader changed from <none> to 829552e4c23a43349356c4c34c784c62 (127.17.110.1). New cstate: current_term: 1 leader_uuid: "829552e4c23a43349356c4c34c784c62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829552e4c23a43349356c4c34c784c62" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 34987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "775ee470852342518a03a533889cc550" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "533163cd8b404049ae59946e598492fa" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36227 } health_report { overall_health: UNKNOWN } } }
I20260807 18:18:45.826313 9254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:45.819700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38188 (local address 127.17.110.1:34987)
0807 18:18:45.819933 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:45.819939 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:45.819996 (+ 57us) server_negotiation.cc:407] Connection header received
0807 18:18:45.820272 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:45.820279 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:45.820391 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:45.820547 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:18:45.820976 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.822095 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.823657 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.824321 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.824447 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:45.824997 (+ 550us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:18:45.825654 (+ 657us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:18:45.825804 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:45.825933 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 18:18:45.826031 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260807 18:18:45.835796 8949 raft_consensus.cc:1261] T 3ab00d0f3a564df5aa430bfa063b25aa P 775ee470852342518a03a533889cc550 [term 1 FOLLOWER]: Refusing update from remote peer 829552e4c23a43349356c4c34c784c62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:18:45.835796 9173 raft_consensus.cc:1261] T 3ab00d0f3a564df5aa430bfa063b25aa P 533163cd8b404049ae59946e598492fa [term 1 FOLLOWER]: Refusing update from remote peer 829552e4c23a43349356c4c34c784c62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:18:45.837203 9277 consensus_queue.cc:1035] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "775ee470852342518a03a533889cc550" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 38123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:18:45.837694 9270 consensus_queue.cc:1035] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "533163cd8b404049ae59946e598492fa" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 36227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:18:45.878196 8526 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:18:45.878541 8526 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:18:45.882289 8526 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af: Sending DeleteTablet for 3 replicas of tablet 3ab00d0f3a564df5aa430bfa063b25aa
I20260807 18:18:45.883831 8718 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ab00d0f3a564df5aa430bfa063b25aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:18:45 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:38182
I20260807 18:18:45.883875 9132 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ab00d0f3a564df5aa430bfa063b25aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:18:45 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:36430
I20260807 18:18:45.883924 8916 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ab00d0f3a564df5aa430bfa063b25aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:18:45 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:33954
I20260807 18:18:45.885113 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 8589
I20260807 18:18:45.912061 9304 tablet_replica.cc:330] T 3ab00d0f3a564df5aa430bfa063b25aa P 533163cd8b404049ae59946e598492fa: stopping tablet replica
I20260807 18:18:45.912778 9304 raft_consensus.cc:2229] T 3ab00d0f3a564df5aa430bfa063b25aa P 533163cd8b404049ae59946e598492fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:18:45.913496 9304 raft_consensus.cc:2258] T 3ab00d0f3a564df5aa430bfa063b25aa P 533163cd8b404049ae59946e598492fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:18:45.921684 9301 tablet_replica.cc:330] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62: stopping tablet replica
I20260807 18:18:45.921657 9304 ts_tablet_manager.cc:1905] T 3ab00d0f3a564df5aa430bfa063b25aa P 533163cd8b404049ae59946e598492fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:18:45.922348 9301 raft_consensus.cc:2229] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:18:45.923406 9301 raft_consensus.cc:2258] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:18:45.924415 9302 tablet_replica.cc:330] T 3ab00d0f3a564df5aa430bfa063b25aa P 775ee470852342518a03a533889cc550: stopping tablet replica
I20260807 18:18:45.924894 9302 raft_consensus.cc:2229] T 3ab00d0f3a564df5aa430bfa063b25aa P 775ee470852342518a03a533889cc550 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:18:45.925508 9302 raft_consensus.cc:2258] T 3ab00d0f3a564df5aa430bfa063b25aa P 775ee470852342518a03a533889cc550 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:18:45.927702 9302 ts_tablet_manager.cc:1905] T 3ab00d0f3a564df5aa430bfa063b25aa P 775ee470852342518a03a533889cc550: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:18:45.927707 9301 ts_tablet_manager.cc:1905] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:18:45.930255 9304 ts_tablet_manager.cc:1918] T 3ab00d0f3a564df5aa430bfa063b25aa P 533163cd8b404049ae59946e598492fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:18:45.930485 9304 log.cc:1196] T 3ab00d0f3a564df5aa430bfa063b25aa P 533163cd8b404049ae59946e598492fa: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/wal/wals/3ab00d0f3a564df5aa430bfa063b25aa
I20260807 18:18:45.931135 9304 ts_tablet_manager.cc:1939] T 3ab00d0f3a564df5aa430bfa063b25aa P 533163cd8b404049ae59946e598492fa: Deleting consensus metadata
I20260807 18:18:45.933317 8505 catalog_manager.cc:4841] TS 533163cd8b404049ae59946e598492fa (127.17.110.3:36227): tablet 3ab00d0f3a564df5aa430bfa063b25aa (table test-table [id=a12321c9933b485e97194967cecabdc7]) successfully deleted
I20260807 18:18:45.935592 9302 ts_tablet_manager.cc:1918] T 3ab00d0f3a564df5aa430bfa063b25aa P 775ee470852342518a03a533889cc550: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:18:45.935817 9302 log.cc:1196] T 3ab00d0f3a564df5aa430bfa063b25aa P 775ee470852342518a03a533889cc550: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/wal/wals/3ab00d0f3a564df5aa430bfa063b25aa
I20260807 18:18:45.936456 9302 ts_tablet_manager.cc:1939] T 3ab00d0f3a564df5aa430bfa063b25aa P 775ee470852342518a03a533889cc550: Deleting consensus metadata
I20260807 18:18:45.938589 8503 catalog_manager.cc:4841] TS 775ee470852342518a03a533889cc550 (127.17.110.2:38123): tablet 3ab00d0f3a564df5aa430bfa063b25aa (table test-table [id=a12321c9933b485e97194967cecabdc7]) successfully deleted
I20260807 18:18:45.939895 9301 ts_tablet_manager.cc:1918] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:18:45.940881 9301 log.cc:1196] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/wal/wals/3ab00d0f3a564df5aa430bfa063b25aa
I20260807 18:18:45.941572 9301 ts_tablet_manager.cc:1939] T 3ab00d0f3a564df5aa430bfa063b25aa P 829552e4c23a43349356c4c34c784c62: Deleting consensus metadata
I20260807 18:18:45.943569 8503 catalog_manager.cc:4841] TS 829552e4c23a43349356c4c34c784c62 (127.17.110.1:34987): tablet 3ab00d0f3a564df5aa430bfa063b25aa (table test-table [id=a12321c9933b485e97194967cecabdc7]) successfully deleted
I20260807 18:18:45.944028 9254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:45.886572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38198 (local address 127.17.110.1:34987)
0807 18:18:45.886864 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:45.886874 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:45.907468 (+ 20594us) server_negotiation.cc:407] Connection header received
0807 18:18:45.908146 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:45.908156 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:45.908292 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:45.908394 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:45.914198 (+ 5804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.916076 (+ 1878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.919593 (+ 3517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:45.920684 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:45.920839 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:45.934494 (+ 13655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:45.934521 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:45.934549 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:45.934610 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:45.939943 (+ 5333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:45.940932 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:45.940942 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:45.940946 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:45.941172 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:45.941801 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:45.941808 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:45.941814 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:45.942334 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:45.942678 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:45.943149 (+ 471us) server_negotiation.cc:299] Negotiation successful
0807 18:18:45.943594 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 18:18:45.944802 8808 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:18:46.051051 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 8589
I20260807 18:18:46.082837 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 8821
I20260807 18:18:46.098363 9257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:46.083767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33966 (local address 127.17.110.2:38123)
0807 18:18:46.084001 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:46.084007 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:46.084056 (+ 49us) server_negotiation.cc:407] Connection header received
0807 18:18:46.084126 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:46.084130 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:46.084203 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:46.084298 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:46.085762 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:46.086527 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:46.087936 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:46.088404 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:46.088509 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:46.091369 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:46.091388 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:46.091407 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:46.091454 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:46.095083 (+ 3629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:46.095971 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:46.095980 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:46.095982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:46.096203 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:46.096621 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:46.096625 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:46.096627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:46.097049 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:46.097350 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:46.097699 (+ 349us) server_negotiation.cc:299] Negotiation successful
0807 18:18:46.097890 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260807 18:18:46.099201 9012 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:18:46.191215 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 8821
I20260807 18:18:46.218011 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 9026
I20260807 18:18:46.235625 9256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:46.219045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36446 (local address 127.17.110.3:36227)
0807 18:18:46.219253 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:46.219257 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:46.219278 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:18:46.219410 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:46.219414 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:46.219507 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:46.219631 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:46.221234 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:46.221948 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:46.223674 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:46.224114 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:46.224262 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:46.227354 (+ 3092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:46.227374 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:46.227393 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:46.227438 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:46.231695 (+ 4257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:46.232999 (+ 1304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:46.233008 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:46.233011 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:46.233194 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:46.233707 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:46.233713 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:46.233717 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:46.234168 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:46.234530 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:46.234939 (+ 409us) server_negotiation.cc:299] Negotiation successful
0807 18:18:46.235207 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 18:18:46.236486 9229 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:18:46.329447 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 9026
I20260807 18:18:46.352916 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 8476
I20260807 18:18:46.354180 8577 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:18:46.470490 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 8476
I20260807 18:19:06.502678 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:45375
--webserver_interface=127.17.110.62
--webserver_port=44325
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:46703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:45375
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:19:06.882404 9313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:19:06.882897 9313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:19:06.883085 9313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:19:06.892077 9313 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 18:19:06.892284 9313 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 18:19:06.892391 9313 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 18:19:06.892495 9313 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:19:06.892609 9313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:19:06.892745 9313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:19:06.892836 9313 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:19:06.892920 9313 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:19:06.903753 9313 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:46703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:45375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:45375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=44325
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:19:06.905282 9313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:19:06.906812 9313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:19:06.920219 9320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:06.920744 9322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:06.921166 9319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:19:06.922482 9313 server_base.cc:1034] running on GCE node
I20260807 18:19:06.923671 9313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:19:06.925437 9313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:19:06.926837 9313 hybrid_clock.cc:648] HybridClock initialized: now 1786126746926750 us; error 50 us; skew 500 ppm
I20260807 18:19:06.927263 9313 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:19:06 dist-test-slave-39l8 krb5kdc[7680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126746, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:19:06.936576 9313 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:19:06.938793 9313 webserver.cc:460] Webserver started at http://127.17.110.62:44325/ using document root <none> and password file <none>
I20260807 18:19:06.939736 9313 fs_manager.cc:359] Metadata directory not provided
I20260807 18:19:06.939917 9313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:19:06.947031 9313 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.003s
I20260807 18:19:06.950588 9332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:06.952527 9313 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:19:06.952806 9313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "7a3b1cd4083644cabeef6647a97bc5af"
format_stamp: "Formatted at 2026-08-07 18:18:20 on dist-test-slave-39l8"
server_key: "c4c13ef066df83212eb28a8223bc959a"
server_key_iv: "ab9f0012e71f5d53b3a51ec2d01a39c1"
server_key_version: "encryptionkey@0"
I20260807 18:19:06.954018 9313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:19:06.986465 9313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:19:06.991791 9313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:19:06.992199 9313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:19:07.016471 9313 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:45375
I20260807 18:19:07.016535 9402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:45375 every 8 connection(s)
I20260807 18:19:07.018342 9313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:19:07.019671 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 9313
I20260807 18:19:07.020787 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.1:34987
--local_ip_for_outbound_sockets=127.17.110.1
--tserver_master_addrs=127.17.110.62:45375
--webserver_port=33745
--webserver_interface=127.17.110.1
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:46703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:19:07.027522 9403 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:19:07.031636 9403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af: Bootstrap starting.
I20260807 18:19:07.055939 9403 log.cc:824] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af: Log is configured to *not* fsync() on all Append() calls
I20260807 18:19:07.075749 9403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:19:07.076439 9403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af: Bootstrap complete.
I20260807 18:19:07.087975 9403 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3b1cd4083644cabeef6647a97bc5af" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45375 } }
I20260807 18:19:07.089124 9403 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a3b1cd4083644cabeef6647a97bc5af, State: Initialized, Role: FOLLOWER
I20260807 18:19:07.089955 9403 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3b1cd4083644cabeef6647a97bc5af" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45375 } }
I20260807 18:19:07.090260 9403 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:19:07.090423 9403 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:19:07.090647 9403 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [term 2 FOLLOWER]: Advancing to term 3
I20260807 18:19:07.095441 9403 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3b1cd4083644cabeef6647a97bc5af" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45375 } }
I20260807 18:19:07.096128 9403 leader_election.cc:302] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a3b1cd4083644cabeef6647a97bc5af; no voters:
I20260807 18:19:07.096962 9403 leader_election.cc:288] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 18:19:07.097150 9407 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [term 3 FOLLOWER]: Leader election won for term 3
I20260807 18:19:07.098492 9407 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [term 3 LEADER]: Becoming Leader. State: Replica: 7a3b1cd4083644cabeef6647a97bc5af, State: Running, Role: LEADER
I20260807 18:19:07.099347 9407 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3b1cd4083644cabeef6647a97bc5af" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45375 } }
I20260807 18:19:07.099766 9403 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:19:07.102536 9411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a3b1cd4083644cabeef6647a97bc5af. Latest consensus state: current_term: 3 leader_uuid: "7a3b1cd4083644cabeef6647a97bc5af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3b1cd4083644cabeef6647a97bc5af" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45375 } } }
I20260807 18:19:07.103093 9411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [sys.catalog]: This master's current role is: LEADER
I20260807 18:19:07.102667 9408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "7a3b1cd4083644cabeef6647a97bc5af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3b1cd4083644cabeef6647a97bc5af" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 45375 } } }
I20260807 18:19:07.103246 9408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af [sys.catalog]: This master's current role is: LEADER
I20260807 18:19:07.116171 9416 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:19:07.127408 9416 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:19:07.129655 9416 catalog_manager.cc:1259] Loaded cluster ID: ab3bf186d2524cd989e7a485c948352c
I20260807 18:19:07.129976 9416 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:19:07.135511 9416 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:19:07.137843 9416 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af: Loaded TSK: 0
I20260807 18:19:07.138171 9416 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7a3b1cd4083644cabeef6647a97bc5af: Loaded TSK: 1
F20260807 18:19:07.139587 9416 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f2732d7a9ad google::LogMessage::Fail() at ??:0
@ 0x7f2732d7d88a google::LogMessage::SendToLog() at ??:0
@ 0x7f2732d7a4de google::LogMessage::Flush() at ??:0
@ 0x7f2732d7ba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
I20260807 18:19:07.154244 9433 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f2736ade7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f2736acd73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f27487a91c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f27487a8783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f27488342a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f27488340c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f274887e5e9 std::function<>::operator()() at ??:0
@ 0x7f27487accdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f27487aac36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f2748832610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f2748832439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f27359c6345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f27359c991c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f27359c9799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f27359a4eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f2738c7e6db start_thread at ??:0
@ 0x7f2731c1461f clone at ??:0
W20260807 18:19:07.494524 9405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:19:07.495023 9405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:19:07.495218 9405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:19:07.503041 9405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:19:07.503242 9405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:19:07.503443 9405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.1
I20260807 18:19:07.515004 9405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:46703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.1:34987
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.110.1
--webserver_port=33745
--tserver_master_addrs=127.17.110.62:45375
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.1
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:19:07.516716 9405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:19:07.518333 9405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:19:07.534915 9469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:07.535285 9472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:07.535633 9470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:19:07.536542 9405 server_base.cc:1034] running on GCE node
I20260807 18:19:07.537357 9405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:19:07.539420 9405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:19:07.540805 9405 hybrid_clock.cc:648] HybridClock initialized: now 1786126747540718 us; error 64 us; skew 500 ppm
I20260807 18:19:07.541356 9405 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:19:07 dist-test-slave-39l8 krb5kdc[7680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126747, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:19:07.545770 9405 init.cc:377] Logged in from keytab as kudu/127.17.110.1@KRBTEST.COM (short username kudu)
I20260807 18:19:07.547703 9405 webserver.cc:460] Webserver started at http://127.17.110.1:33745/ using document root <none> and password file <none>
I20260807 18:19:07.548555 9405 fs_manager.cc:359] Metadata directory not provided
I20260807 18:19:07.548730 9405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:19:07.555606 9405 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260807 18:19:07.559219 9480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:07.560840 9405 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260807 18:19:07.561089 9405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/wal
uuid: "829552e4c23a43349356c4c34c784c62"
format_stamp: "Formatted at 2026-08-07 18:18:20 on dist-test-slave-39l8"
server_key: "dcc5621c19c74d9c7e41f6c9fd89a00f"
server_key_iv: "29553daaa980ebbd0043c8251d86ee53"
server_key_version: "encryptionkey@0"
I20260807 18:19:07.562306 9405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:19:07.585836 9405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:19:07.591374 9405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:19:07.591743 9405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:19:07.593063 9405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:19:07.595536 9405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:19:07.595916 9405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:07.596089 9405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:19:07.596208 9405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:07.639533 9405 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.1:34987
I20260807 18:19:07.639572 9647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.1:34987 every 8 connection(s)
I20260807 18:19:07.641242 9405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
I20260807 18:19:07.647583 9651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:07.644956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.1:44981)
0807 18:19:07.645978 (+ 1022us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:07.646267 (+ 289us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":395,"threads_started":1}
W20260807 18:19:07.648687 9648 heartbeater.cc:644] Failed to heartbeat to 127.17.110.62:45375 (0 consecutive failures): Network error: Failed to ping master at 127.17.110.62:45375: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
I20260807 18:19:07.649999 9651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:07.649510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.1:33731)
0807 18:19:07.649758 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:07.649856 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260807 18:19:07.651117 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 9405
I20260807 18:19:07.651489 9651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:07.650903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.1:37013)
0807 18:19:07.651106 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:07.651310 (+ 204us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 18:19:07.651945 9648 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 18:19:07.652076 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.2:38123
--local_ip_for_outbound_sockets=127.17.110.2
--tserver_master_addrs=127.17.110.62:45375
--webserver_port=43341
--webserver_interface=127.17.110.2
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:46703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:19:08.068004 9654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:19:08.068369 9654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:19:08.068504 9654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:19:08.076290 9654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:19:08.076462 9654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:19:08.076604 9654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.2
I20260807 18:19:08.087039 9654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:46703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.2:38123
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.110.2
--webserver_port=43341
--tserver_master_addrs=127.17.110.62:45375
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.2
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:19:08.088492 9654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:19:08.089879 9654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:19:08.106323 9666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:08.106388 9662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:08.107277 9670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:19:08.108139 9654 server_base.cc:1034] running on GCE node
I20260807 18:19:08.109236 9654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:19:08.110553 9654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:19:08.111860 9654 hybrid_clock.cc:648] HybridClock initialized: now 1786126748111757 us; error 95 us; skew 500 ppm
I20260807 18:19:08.112349 9654 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:19:08 dist-test-slave-39l8 krb5kdc[7680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126748, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:19:08.118592 9654 init.cc:377] Logged in from keytab as kudu/127.17.110.2@KRBTEST.COM (short username kudu)
I20260807 18:19:08.120916 9654 webserver.cc:460] Webserver started at http://127.17.110.2:43341/ using document root <none> and password file <none>
I20260807 18:19:08.121840 9654 fs_manager.cc:359] Metadata directory not provided
I20260807 18:19:08.122035 9654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:19:08.129627 9654 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 18:19:08.133658 9679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:08.135861 9654 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 18:19:08.136124 9654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/wal
uuid: "775ee470852342518a03a533889cc550"
format_stamp: "Formatted at 2026-08-07 18:18:21 on dist-test-slave-39l8"
server_key: "345d8a4e5013cf01743d32339088b2d4"
server_key_iv: "66e985761c60acea9673dba49f81f3ff"
server_key_version: "encryptionkey@0"
I20260807 18:19:08.137305 9654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:19:08.165623 9654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:19:08.171105 9654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:19:08.171427 9654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:19:08.172742 9654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:19:08.174347 9654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:19:08.174499 9654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:08.174659 9654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:19:08.174768 9654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:08.215855 9654 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.2:38123
I20260807 18:19:08.215962 9838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.2:38123 every 8 connection(s)
I20260807 18:19:08.217521 9654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
I20260807 18:19:08.220811 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 9654
I20260807 18:19:08.221669 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.3:36227
--local_ip_for_outbound_sockets=127.17.110.3
--tserver_master_addrs=127.17.110.62:45375
--webserver_port=43873
--webserver_interface=127.17.110.3
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:46703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:19:08.223665 9841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:08.221312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.2:52965)
0807 18:19:08.222329 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:08.222623 (+ 294us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":663,"thread_start_us":408,"threads_started":1}
W20260807 18:19:08.224459 9839 heartbeater.cc:644] Failed to heartbeat to 127.17.110.62:45375 (0 consecutive failures): Network error: Failed to ping master at 127.17.110.62:45375: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
I20260807 18:19:08.225446 9841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:08.225056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.2:55311)
0807 18:19:08.225243 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:08.225328 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 18:19:08.226568 9841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:08.226065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.2:44955)
0807 18:19:08.226274 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:08.226395 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260807 18:19:08.226931 9839 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:19:08.616168 9842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:19:08.616523 9842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:19:08.616701 9842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:19:08.624853 9842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:19:08.625010 9842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:19:08.625238 9842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.3
I20260807 18:19:08.636045 9842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:46703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.3:36227
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.110.3
--webserver_port=43873
--tserver_master_addrs=127.17.110.62:45375
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.3
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:19:08.637965 9842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:19:08.641032 9842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:19:08.653635 9853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:08.652682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.1:33225)
0807 18:19:08.653264 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:08.653389 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":295,"threads_started":1}
W20260807 18:19:08.658942 9848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:08.658990 9851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:08.658890 9849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:19:08.665012 9842 server_base.cc:1034] running on GCE node
I20260807 18:19:08.665942 9842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:19:08.667430 9842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:19:08.668947 9842 hybrid_clock.cc:648] HybridClock initialized: now 1786126748668874 us; error 56 us; skew 500 ppm
I20260807 18:19:08.669516 9842 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:19:08 dist-test-slave-39l8 krb5kdc[7680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126748, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:19:08.673840 9842 init.cc:377] Logged in from keytab as kudu/127.17.110.3@KRBTEST.COM (short username kudu)
I20260807 18:19:08.676587 9842 webserver.cc:460] Webserver started at http://127.17.110.3:43873/ using document root <none> and password file <none>
I20260807 18:19:08.677734 9842 fs_manager.cc:359] Metadata directory not provided
I20260807 18:19:08.677875 9842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:19:08.685168 9842 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 18:19:08.689870 9867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:08.692058 9842 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 18:19:08.692329 9842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/wal
uuid: "533163cd8b404049ae59946e598492fa"
format_stamp: "Formatted at 2026-08-07 18:18:21 on dist-test-slave-39l8"
server_key: "550f862ace80c150582d7ab6bead54f5"
server_key_iv: "ce733b93103111465bb94eabed1ccfd6"
server_key_version: "encryptionkey@0"
I20260807 18:19:08.693516 9842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:19:08.711918 9842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:19:08.717303 9842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:19:08.717629 9842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:19:08.718809 9842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:19:08.720316 9842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:19:08.720410 9842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:08.720577 9842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:19:08.720669 9842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:08.759049 9842 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.3:36227
I20260807 18:19:08.759104 10024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.3:36227 every 8 connection(s)
I20260807 18:19:08.760807 9842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
I20260807 18:19:08.767156 10029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:08.764916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.3:36077)
0807 18:19:08.765791 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:08.766048 (+ 257us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":326,"threads_started":1}
W20260807 18:19:08.768050 10025 heartbeater.cc:644] Failed to heartbeat to 127.17.110.62:45375 (0 consecutive failures): Network error: Failed to ping master at 127.17.110.62:45375: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
I20260807 18:19:08.768409 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 9842
I20260807 18:19:08.769193 10029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:08.768767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.3:36195)
0807 18:19:08.768970 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:08.769067 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260807 18:19:08.770589 10029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:08.770152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.3:52509)
0807 18:19:08.770347 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:08.770450 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260807 18:19:08.770879 10025 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 18:19:09.228694 10031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:09.227725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.2:46649)
0807 18:19:09.228326 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:09.228435 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":299,"threads_started":1}
I20260807 18:19:09.655509 10033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:09.654623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.1:60269)
0807 18:19:09.655240 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:09.655328 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":312,"threads_started":1}
I20260807 18:19:09.772606 10034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:09.771607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.3:43469)
0807 18:19:09.772171 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:09.772333 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":269,"threads_started":1}
I20260807 18:19:10.231164 10035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:10.230012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.2:33755)
0807 18:19:10.230848 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:10.230965 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":431,"threads_started":1}
I20260807 18:19:10.657915 10036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:10.656773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.1:50793)
0807 18:19:10.657462 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:10.657602 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":322,"threads_started":1}
I20260807 18:19:10.774940 10037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:10.773953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.3:48265)
0807 18:19:10.774559 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:10.774660 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":328,"threads_started":1}
I20260807 18:19:11.233171 10038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:11.232282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.2:59409)
0807 18:19:11.232855 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:11.232958 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":354,"threads_started":1}
I20260807 18:19:11.660338 10041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:11.659373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.1:54975)
0807 18:19:11.660007 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:11.660148 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":357,"threads_started":1}
I20260807 18:19:11.777372 10044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:11.776276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.3:51047)
0807 18:19:11.777063 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:11.777174 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":454,"threads_started":1}
I20260807 18:19:12.235495 10047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:12.234397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.2:59919)
0807 18:19:12.235184 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:12.235289 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":470,"threads_started":1}
I20260807 18:19:12.662321 10048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:12.661462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.1:49183)
0807 18:19:12.662052 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:12.662139 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":318,"threads_started":1}
I20260807 18:19:12.779595 10049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:12.778640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.3:44887)
0807 18:19:12.779275 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:12.779362 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":355,"threads_started":1}
I20260807 18:19:13.237534 10051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:13.236597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.2:58991)
0807 18:19:13.237250 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:13.237349 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":310,"threads_started":1}
I20260807 18:19:13.664530 10052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:13.663499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.1:43323)
0807 18:19:13.664168 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:13.664299 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":370,"threads_started":1}
I20260807 18:19:13.781518 10053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:13.780745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.3:53607)
0807 18:19:13.781256 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:13.781343 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":262,"threads_started":1}
I20260807 18:19:14.239645 10054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:14.238676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.2:37539)
0807 18:19:14.239326 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:14.239435 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":302,"threads_started":1}
I20260807 18:19:14.666854 10056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:14.665653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.1:43853)
0807 18:19:14.666324 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:14.666466 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":246,"threads_started":1}
I20260807 18:19:14.783906 10057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:14.782985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.3:44761)
0807 18:19:14.783603 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:14.783719 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":350,"threads_started":1}
I20260807 18:19:15.241544 10058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:15.240642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.2:37519)
0807 18:19:15.241218 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:15.241320 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":266,"threads_started":1}
I20260807 18:19:15.669008 10059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:15.668159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.1:35503)
0807 18:19:15.668744 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:15.668836 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":268,"threads_started":1}
I20260807 18:19:15.785921 10060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:15.785059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.3:59435)
0807 18:19:15.785607 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:15.785704 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":236,"threads_started":1}
I20260807 18:19:16.243671 10061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:16.242685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.2:45217)
0807 18:19:16.243402 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:16.243495 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":435,"threads_started":1}
I20260807 18:19:16.671151 10064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:16.670156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.1:42273)
0807 18:19:16.670844 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:16.670944 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":295,"threads_started":1}
I20260807 18:19:16.788100 10065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:16.787264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.3:36319)
0807 18:19:16.787774 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:16.787909 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":259,"threads_started":1}
I20260807 18:19:17.245486 10067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:17.244650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.2:52141)
0807 18:19:17.245196 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:17.245291 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":266,"threads_started":1}
I20260807 18:19:17.673167 10068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:17.672316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.1:39099)
0807 18:19:17.672840 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:17.672930 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":307,"threads_started":1}
I20260807 18:19:17.790169 10070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:17.789337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.3:54865)
0807 18:19:17.789875 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:17.789977 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":255,"threads_started":1}
I20260807 18:19:18.247457 10071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:18.246616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.2:53939)
0807 18:19:18.247189 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:18.247283 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":335,"threads_started":1}
I20260807 18:19:18.675032 10072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:18.674227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.1:57533)
0807 18:19:18.674745 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:18.674848 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":234,"threads_started":1}
I20260807 18:19:18.792194 10074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:18.791367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.3:56267)
0807 18:19:18.791874 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:18.791975 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":277,"threads_started":1}
I20260807 18:19:19.249542 10075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:19.248712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.2:39463)
0807 18:19:19.249236 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:19.249349 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":270,"threads_started":1}
I20260807 18:19:19.677263 10076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:19.676152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.1:58107)
0807 18:19:19.676758 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:19.676853 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":278,"threads_started":1}
I20260807 18:19:19.794008 10077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:19.793220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.3:53501)
0807 18:19:19.793723 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:19.793820 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":287,"threads_started":1}
I20260807 18:19:20.251669 10078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:20.250789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.2:38379)
0807 18:19:20.251390 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:20.251490 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":342,"threads_started":1}
I20260807 18:19:20.679528 10079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:20.678577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.1:43623)
0807 18:19:20.679222 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:20.679336 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":304,"threads_started":1}
I20260807 18:19:20.795931 10080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:20.795197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.3:60119)
0807 18:19:20.795686 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:20.795771 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":258,"threads_started":1}
I20260807 18:19:21.253912 10081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:21.252861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.2:37225)
0807 18:19:21.253537 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:21.253675 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":287,"threads_started":1}
I20260807 18:19:21.681690 10082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:21.680761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.1:48291)
0807 18:19:21.681331 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:21.681432 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":295,"threads_started":1}
I20260807 18:19:21.798163 10084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:21.797068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.3:33431)
0807 18:19:21.797817 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:21.797939 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":372,"threads_started":1}
I20260807 18:19:22.255911 10085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:22.255044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.2:53537)
0807 18:19:22.255617 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:22.255723 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":297,"threads_started":1}
I20260807 18:19:22.683998 10086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:22.683136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.1:50349)
0807 18:19:22.683682 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:22.683786 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":276,"threads_started":1}
I20260807 18:19:22.800204 10087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:22.799350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.3:46091)
0807 18:19:22.799880 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:22.799981 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":267,"threads_started":1}
I20260807 18:19:23.258203 10089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:23.257293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.2:54109)
0807 18:19:23.257817 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:23.257939 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":275,"threads_started":1}
I20260807 18:19:23.685994 10090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:23.685190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.1:34865)
0807 18:19:23.685728 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:23.685826 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":262,"threads_started":1}
I20260807 18:19:23.773609 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 9405
I20260807 18:19:23.796337 10093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:23.775392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42360 (local address 127.17.110.1:34987)
0807 18:19:23.776101 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:23.776115 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:23.776166 (+ 51us) server_negotiation.cc:407] Connection header received
0807 18:19:23.777850 (+ 1684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:23.777862 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:23.778322 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:23.778656 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:23.780378 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:23.781717 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:23.783350 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:23.784186 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:23.784367 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:23.788280 (+ 3913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:23.788292 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:23.788313 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:19:23.788375 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:23.791827 (+ 3452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:23.792917 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:23.792930 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:23.792938 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:23.793168 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:23.793665 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:23.793670 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:23.793673 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:23.794427 (+ 754us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:23.794828 (+ 401us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:23.795404 (+ 576us) server_negotiation.cc:299] Negotiation successful
0807 18:19:23.795974 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":358,"threads_started":1}
I20260807 18:19:23.798010 9642 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:19:23.803047 10097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:23.801849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:45375 (local address 127.17.110.3:37763)
0807 18:19:23.802546 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:23.802685 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.62:45375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":340,"threads_started":1}
I20260807 18:19:23.885634 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 9405
I20260807 18:19:23.916153 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 9654
I20260807 18:19:23.937105 10100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:23.917253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36672 (local address 127.17.110.2:38123)
0807 18:19:23.917800 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:23.917815 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:23.917857 (+ 42us) server_negotiation.cc:407] Connection header received
0807 18:19:23.919596 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:23.919608 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:23.920104 (+ 496us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:23.920526 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:23.922169 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:23.923639 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:23.925091 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:23.925908 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:23.926141 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:23.929232 (+ 3091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:23.929242 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:23.929262 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:19:23.929317 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:23.932999 (+ 3682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:23.933979 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:23.933988 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:23.933997 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:23.934212 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:23.934672 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:23.934677 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:23.934679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:23.935343 (+ 664us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:23.935698 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:23.936284 (+ 586us) server_negotiation.cc:299] Negotiation successful
0807 18:19:23.936772 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":258,"threads_started":1}
I20260807 18:19:23.938930 9834 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:19:24.029976 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 9654
I20260807 18:19:24.059000 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 9842
I20260807 18:19:24.081408 10103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:24.060289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60952 (local address 127.17.110.3:36227)
0807 18:19:24.060937 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:24.060955 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:24.061005 (+ 50us) server_negotiation.cc:407] Connection header received
0807 18:19:24.063190 (+ 2185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:24.063206 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:24.063723 (+ 517us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:24.064160 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:24.065936 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:24.067536 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:24.069242 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:24.070101 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:24.070348 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:24.073622 (+ 3274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:24.073633 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:24.073654 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:19:24.073715 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:24.077308 (+ 3593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:24.078266 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:24.078275 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:24.078284 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:24.078494 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:24.078995 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:24.079007 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:24.079010 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:24.079668 (+ 658us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:24.080032 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:24.080544 (+ 512us) server_negotiation.cc:299] Negotiation successful
0807 18:19:24.081045 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":288,"threads_started":1}
I20260807 18:19:24.082986 10019 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:19:24.169401 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 9842
2026-08-07T18:19:24Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (66230 ms)
[----------] 30 tests from SecurityITest (334536 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:19:24 dist-test-slave-39l8 krb5kdc[10107](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:19:24 dist-test-slave-39l8 krb5kdc[10107](info): set up 2 sockets
Aug 07 18:19:24 dist-test-slave-39l8 krb5kdc[10107](info): commencing operation
krb5kdc: starting...
W20260807 18:19:25.539477 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.320s user 0.002s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:19:25 dist-test-slave-39l8 krb5kdc[10107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:19:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:19:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.110.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
I20260807 18:19:25.894037 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:46857
--webserver_interface=127.17.110.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:33471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:46857
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:19:26.275885 10127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:19:26.276245 10127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:19:26.276383 10127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:19:26.284170 10127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:19:26.284315 10127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:19:26.284407 10127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:19:26.284503 10127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:19:26.284590 10127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:19:26.295457 10127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:33471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:46857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:46857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:19:26.296988 10127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:19:26.298467 10127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:19:26.312947 10132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:26.312911 10139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:26.312875 10133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:19:26.313911 10127 server_base.cc:1034] running on GCE node
I20260807 18:19:26.314968 10127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:19:26.316802 10127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:19:26.318097 10127 hybrid_clock.cc:648] HybridClock initialized: now 1786126766318015 us; error 69 us; skew 500 ppm
I20260807 18:19:26.318634 10127 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:19:26 dist-test-slave-39l8 krb5kdc[10107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126766, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:19:26.327136 10127 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:19:26.329249 10127 webserver.cc:460] Webserver started at http://127.17.110.62:38033/ using document root <none> and password file <none>
I20260807 18:19:26.330132 10127 fs_manager.cc:359] Metadata directory not provided
I20260807 18:19:26.330307 10127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:19:26.330762 10127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:19:26.334439 10127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "00a893209cf644609c81e01b2f7dbad3"
format_stamp: "Formatted at 2026-08-07 18:19:26 on dist-test-slave-39l8"
server_key: "2af0b457e19a8ab4ebe9c5643d290e35"
server_key_iv: "bb3e5f14f968ca21c15da5783fc38b9c"
server_key_version: "encryptionkey@0"
I20260807 18:19:26.335498 10127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "00a893209cf644609c81e01b2f7dbad3"
format_stamp: "Formatted at 2026-08-07 18:19:26 on dist-test-slave-39l8"
server_key: "2af0b457e19a8ab4ebe9c5643d290e35"
server_key_iv: "bb3e5f14f968ca21c15da5783fc38b9c"
server_key_version: "encryptionkey@0"
I20260807 18:19:26.342027 10127 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 18:19:26.346534 10152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:26.348309 10127 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:19:26.348575 10127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "00a893209cf644609c81e01b2f7dbad3"
format_stamp: "Formatted at 2026-08-07 18:19:26 on dist-test-slave-39l8"
server_key: "2af0b457e19a8ab4ebe9c5643d290e35"
server_key_iv: "bb3e5f14f968ca21c15da5783fc38b9c"
server_key_version: "encryptionkey@0"
I20260807 18:19:26.348855 10127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:19:26.368166 10127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:19:26.373622 10127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:19:26.373996 10127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:19:26.397241 10127 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:46857
I20260807 18:19:26.397246 10228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:46857 every 8 connection(s)
I20260807 18:19:26.399103 10127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:19:26.399775 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 10127
I20260807 18:19:26.400031 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:19:26.401077 17848 external_mini_cluster.cc:1428] Setting key 00da9e7dcbb0a09ec1c3ef4e1703241f
I20260807 18:19:26.405033 10229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 18:19:26 dist-test-slave-39l8 krb5kdc[10107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:19:26.421105 10229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00a893209cf644609c81e01b2f7dbad3: Bootstrap starting.
I20260807 18:19:26.426367 10229 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00a893209cf644609c81e01b2f7dbad3: Neither blocks nor log segments found. Creating new log.
I20260807 18:19:26.427981 10229 log.cc:824] T 00000000000000000000000000000000 P 00a893209cf644609c81e01b2f7dbad3: Log is configured to *not* fsync() on all Append() calls
I20260807 18:19:26.432317 10229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00a893209cf644609c81e01b2f7dbad3: No bootstrap required, opened a new log
I20260807 18:19:26.432808 10235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:26.403523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33654 (local address 127.17.110.62:46857)
0807 18:19:26.404539 (+ 1016us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:26.404560 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:26.404623 (+ 63us) server_negotiation.cc:407] Connection header received
0807 18:19:26.407392 (+ 2769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:26.407429 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:26.408061 (+ 632us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:26.408630 (+ 569us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:26.410327 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:26.412839 (+ 2512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:26.414598 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:26.415855 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:26.416107 (+ 252us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:26.421696 (+ 5589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:26.421711 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:26.421738 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:19:26.421803 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:26.426884 (+ 5081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:26.427902 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:26.427916 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:26.427928 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:26.428242 (+ 314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:26.428914 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:26.428921 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:26.428925 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:26.429619 (+ 694us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:26.430021 (+ 402us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:26.430635 (+ 614us) server_negotiation.cc:299] Negotiation successful
0807 18:19:26.431234 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":686,"thread_start_us":402,"threads_started":1}
I20260807 18:19:26.442085 10229 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00a893209cf644609c81e01b2f7dbad3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00a893209cf644609c81e01b2f7dbad3" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 46857 } }
I20260807 18:19:26.442694 10229 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00a893209cf644609c81e01b2f7dbad3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:19:26.442912 10229 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00a893209cf644609c81e01b2f7dbad3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00a893209cf644609c81e01b2f7dbad3, State: Initialized, Role: FOLLOWER
I20260807 18:19:26.443706 10229 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00a893209cf644609c81e01b2f7dbad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00a893209cf644609c81e01b2f7dbad3" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 46857 } }
I20260807 18:19:26.444002 10229 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00a893209cf644609c81e01b2f7dbad3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:19:26.444170 10229 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00a893209cf644609c81e01b2f7dbad3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:19:26.444367 10229 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00a893209cf644609c81e01b2f7dbad3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:19:26.447075 10229 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00a893209cf644609c81e01b2f7dbad3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00a893209cf644609c81e01b2f7dbad3" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 46857 } }
I20260807 18:19:26.447746 10229 leader_election.cc:302] T 00000000000000000000000000000000 P 00a893209cf644609c81e01b2f7dbad3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00a893209cf644609c81e01b2f7dbad3; no voters:
I20260807 18:19:26.448645 10229 leader_election.cc:288] T 00000000000000000000000000000000 P 00a893209cf644609c81e01b2f7dbad3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:19:26.448835 10237 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00a893209cf644609c81e01b2f7dbad3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:19:26.449957 10237 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00a893209cf644609c81e01b2f7dbad3 [term 1 LEADER]: Becoming Leader. State: Replica: 00a893209cf644609c81e01b2f7dbad3, State: Running, Role: LEADER
I20260807 18:19:26.450665 10237 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00a893209cf644609c81e01b2f7dbad3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00a893209cf644609c81e01b2f7dbad3" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 46857 } }
I20260807 18:19:26.451560 10229 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00a893209cf644609c81e01b2f7dbad3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:19:26.454560 10241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00a893209cf644609c81e01b2f7dbad3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00a893209cf644609c81e01b2f7dbad3. Latest consensus state: current_term: 1 leader_uuid: "00a893209cf644609c81e01b2f7dbad3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00a893209cf644609c81e01b2f7dbad3" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 46857 } } }
I20260807 18:19:26.454639 10240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00a893209cf644609c81e01b2f7dbad3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00a893209cf644609c81e01b2f7dbad3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00a893209cf644609c81e01b2f7dbad3" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 46857 } } }
I20260807 18:19:26.455093 10241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00a893209cf644609c81e01b2f7dbad3 [sys.catalog]: This master's current role is: LEADER
I20260807 18:19:26.455134 10240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00a893209cf644609c81e01b2f7dbad3 [sys.catalog]: This master's current role is: LEADER
I20260807 18:19:26.458101 10249 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:19:26.464887 10249 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:19:26.475698 10249 catalog_manager.cc:1347] Generated new cluster ID: f6d5a68d6fe14140a0d11a987b1872b5
I20260807 18:19:26.475878 10249 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:19:26.494092 10249 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:19:26.495709 10249 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:19:26.508998 10249 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00a893209cf644609c81e01b2f7dbad3: Generated new TSK 0
I20260807 18:19:26.510170 10249 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.110.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
I20260807 18:19:26.588905 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.1:0
--local_ip_for_outbound_sockets=127.17.110.1
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:46857
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:33471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:19:26.970312 10271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:19:26.970809 10271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:19:26.970924 10271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:19:26.978570 10271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:19:26.978785 10271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:19:26.978929 10271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.1
I20260807 18:19:26.989753 10271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:33471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:46857
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.1
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:19:26.991281 10271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:19:26.992709 10271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:19:27.007861 10280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:27.007445 10281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:27.008021 10283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:19:27.009526 10271 server_base.cc:1034] running on GCE node
I20260807 18:19:27.010471 10271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:19:27.011910 10271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:19:27.013217 10271 hybrid_clock.cc:648] HybridClock initialized: now 1786126767013158 us; error 51 us; skew 500 ppm
I20260807 18:19:27.013724 10271 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:19:27 dist-test-slave-39l8 krb5kdc[10107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126767, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:19:27.017596 10271 init.cc:377] Logged in from keytab as kudu/127.17.110.1@KRBTEST.COM (short username kudu)
I20260807 18:19:27.019547 10271 webserver.cc:460] Webserver started at http://127.17.110.1:40881/ using document root <none> and password file <none>
I20260807 18:19:27.020366 10271 fs_manager.cc:359] Metadata directory not provided
I20260807 18:19:27.020494 10271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:19:27.020797 10271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:19:27.024621 10271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-0/data/instance:
uuid: "ef37ccc01ac7442d9a80b178d5251e68"
format_stamp: "Formatted at 2026-08-07 18:19:27 on dist-test-slave-39l8"
server_key: "e57f0a55206e14d1c32e52c01f92d4ba"
server_key_iv: "6c5d3496874796c375b5a954b69ef4f7"
server_key_version: "encryptionkey@0"
I20260807 18:19:27.025621 10271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance:
uuid: "ef37ccc01ac7442d9a80b178d5251e68"
format_stamp: "Formatted at 2026-08-07 18:19:27 on dist-test-slave-39l8"
server_key: "e57f0a55206e14d1c32e52c01f92d4ba"
server_key_iv: "6c5d3496874796c375b5a954b69ef4f7"
server_key_version: "encryptionkey@0"
I20260807 18:19:27.032665 10271 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260807 18:19:27.037125 10292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:27.038931 10271 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:19:27.039120 10271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-0/wal
uuid: "ef37ccc01ac7442d9a80b178d5251e68"
format_stamp: "Formatted at 2026-08-07 18:19:27 on dist-test-slave-39l8"
server_key: "e57f0a55206e14d1c32e52c01f92d4ba"
server_key_iv: "6c5d3496874796c375b5a954b69ef4f7"
server_key_version: "encryptionkey@0"
I20260807 18:19:27.039378 10271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:19:27.060343 10271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:19:27.065279 10271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:19:27.065575 10271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:19:27.066692 10271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:19:27.068336 10271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:19:27.068420 10271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:27.068514 10271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:19:27.068559 10271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:27.108592 10271 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.1:43871
I20260807 18:19:27.108664 10457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.1:43871 every 8 connection(s)
I20260807 18:19:27.110183 10271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
I20260807 18:19:27.114221 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 10271
I20260807 18:19:27.114503 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance
I20260807 18:19:27.115373 17848 external_mini_cluster.cc:1428] Setting key cf55207f0a443efbe90478ea35b8fe90
Aug 07 18:19:27 dist-test-slave-39l8 krb5kdc[10107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126767, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:19:27.137689 10462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:27.113435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:40555 (local address 127.17.110.62:46857)
0807 18:19:27.114048 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:27.114054 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:27.115253 (+ 1199us) server_negotiation.cc:407] Connection header received
0807 18:19:27.117512 (+ 2259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:27.117517 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:27.117589 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:27.117759 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:27.119575 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:27.120954 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:27.123242 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:27.124066 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:27.124255 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:27.129331 (+ 5076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:27.129339 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:27.129343 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:19:27.129389 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:27.132701 (+ 3312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:27.134052 (+ 1351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:27.134062 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:27.134069 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:27.134285 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:27.134985 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:27.134994 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:27.135001 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:27.135527 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:27.135721 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:27.136780 (+ 1059us) server_negotiation.cc:299] Negotiation successful
0807 18:19:27.137136 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":321,"threads_started":1}
I20260807 18:19:27.138441 10463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:27.114032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:46857 (local address 127.17.110.1:40555)
0807 18:19:27.115037 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:27.115090 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:27.117157 (+ 2067us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:27.118016 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:27.118026 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:27.118607 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:19:27.119353 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:27.119374 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:27.121353 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:27.121357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:27.123017 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:27.123035 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:27.124377 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:27.124383 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:27.124640 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:27.125742 (+ 1102us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:19:27.125777 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:19:27.128859 (+ 3082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:19:27.132951 (+ 4092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:19:27.132964 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:19:27.132989 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:19:27.133825 (+ 836us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:19:27.134470 (+ 645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:19:27.134474 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:19:27.134477 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:19:27.134829 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:19:27.135717 (+ 888us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:19:27.135726 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:19:27.136222 (+ 496us) client_negotiation.cc:769] Sending connection context
0807 18:19:27.136532 (+ 310us) client_negotiation.cc:240] Negotiation successful
0807 18:19:27.137067 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":378,"threads_started":1}
I20260807 18:19:27.140512 10458 heartbeater.cc:347] Connected to a master server at 127.17.110.62:46857
I20260807 18:19:27.140888 10458 heartbeater.cc:464] Registering TS with master...
I20260807 18:19:27.141762 10458 heartbeater.cc:511] Master 127.17.110.62:46857 requested a full tablet report, sending...
I20260807 18:19:27.144695 10173 ts_manager.cc:194] Registered new tserver with Master: ef37ccc01ac7442d9a80b178d5251e68 (127.17.110.1:43871)
I20260807 18:19:27.146396 10173 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.1@KRBTEST.COM'} at 127.17.110.1:40555
WARNING: no policy specified for kudu/127.17.110.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
I20260807 18:19:27.195755 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.2:0
--local_ip_for_outbound_sockets=127.17.110.2
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:46857
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:33471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:19:27.595227 10469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:19:27.595599 10469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:19:27.595777 10469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:19:27.604076 10469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:19:27.604270 10469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:19:27.604453 10469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.2
I20260807 18:19:27.615338 10469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:33471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:46857
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.2
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:19:27.616837 10469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:19:27.618273 10469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:19:27.633275 10476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:27.633245 10477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:27.633313 10484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:19:27.634343 10469 server_base.cc:1034] running on GCE node
I20260807 18:19:27.635172 10469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:19:27.636247 10469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:19:27.637552 10469 hybrid_clock.cc:648] HybridClock initialized: now 1786126767637495 us; error 50 us; skew 500 ppm
I20260807 18:19:27.638000 10469 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:19:27 dist-test-slave-39l8 krb5kdc[10107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126767, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:19:27.641896 10469 init.cc:377] Logged in from keytab as kudu/127.17.110.2@KRBTEST.COM (short username kudu)
I20260807 18:19:27.644019 10469 webserver.cc:460] Webserver started at http://127.17.110.2:34313/ using document root <none> and password file <none>
I20260807 18:19:27.644837 10469 fs_manager.cc:359] Metadata directory not provided
I20260807 18:19:27.644959 10469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:19:27.645273 10469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:19:27.648568 10469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-1/data/instance:
uuid: "de1af49afae24218968d33e1d584d8b6"
format_stamp: "Formatted at 2026-08-07 18:19:27 on dist-test-slave-39l8"
server_key: "1bfe9958e7a150bb8c320755abd2a9f0"
server_key_iv: "6162c925d5b9d1670d6643256bd4991c"
server_key_version: "encryptionkey@0"
I20260807 18:19:27.649391 10469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance:
uuid: "de1af49afae24218968d33e1d584d8b6"
format_stamp: "Formatted at 2026-08-07 18:19:27 on dist-test-slave-39l8"
server_key: "1bfe9958e7a150bb8c320755abd2a9f0"
server_key_iv: "6162c925d5b9d1670d6643256bd4991c"
server_key_version: "encryptionkey@0"
I20260807 18:19:27.655893 10469 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 18:19:27.660537 10497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:27.662352 10469 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:19:27.662629 10469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-1/wal
uuid: "de1af49afae24218968d33e1d584d8b6"
format_stamp: "Formatted at 2026-08-07 18:19:27 on dist-test-slave-39l8"
server_key: "1bfe9958e7a150bb8c320755abd2a9f0"
server_key_iv: "6162c925d5b9d1670d6643256bd4991c"
server_key_version: "encryptionkey@0"
I20260807 18:19:27.662985 10469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:19:27.690253 10469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:19:27.695425 10469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:19:27.695780 10469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:19:27.696880 10469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:19:27.698423 10469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:19:27.698539 10469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:27.698729 10469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:19:27.698814 10469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260807 18:19:27.736299 10469 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.2:40261
I20260807 18:19:27.736398 10664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.2:40261 every 8 connection(s)
I20260807 18:19:27.737741 10469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
I20260807 18:19:27.745456 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 10469
I20260807 18:19:27.745630 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance
I20260807 18:19:27.746148 17848 external_mini_cluster.cc:1428] Setting key 31d4b372cd8b7a91a6182d7f81f883da
Aug 07 18:19:27 dist-test-slave-39l8 krb5kdc[10107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126767, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:19:27.762277 10668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:27.740765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:54069 (local address 127.17.110.62:46857)
0807 18:19:27.741316 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:27.741321 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:27.742448 (+ 1127us) server_negotiation.cc:407] Connection header received
0807 18:19:27.744815 (+ 2367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:27.744821 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:27.744930 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:27.745079 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:27.747092 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:27.747889 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:27.749573 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:27.750037 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:27.750190 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:27.754497 (+ 4307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:27.754504 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:27.754507 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:19:27.754546 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:27.757882 (+ 3336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:27.759223 (+ 1341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:27.759231 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:27.759237 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:27.759428 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:27.759977 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:27.759986 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:27.759993 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:27.760487 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:27.760694 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:27.761531 (+ 837us) server_negotiation.cc:299] Negotiation successful
0807 18:19:27.761862 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":315,"threads_started":1}
I20260807 18:19:27.763581 10669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:27.741404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:46857 (local address 127.17.110.2:54069)
0807 18:19:27.742206 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:27.742281 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:27.744463 (+ 2182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:27.745395 (+ 932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:27.745404 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:27.746109 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:19:27.746878 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:27.746901 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:27.748074 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:27.748078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:27.749392 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:27.749406 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:27.750315 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:27.750323 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:27.750568 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:27.751446 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:19:27.751471 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:19:27.754188 (+ 2717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:19:27.758102 (+ 3914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:19:27.758114 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:19:27.758144 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:19:27.758989 (+ 845us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:19:27.759580 (+ 591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:19:27.759584 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:19:27.759587 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:19:27.759832 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:19:27.760673 (+ 841us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:19:27.760681 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:19:27.761058 (+ 377us) client_negotiation.cc:769] Sending connection context
0807 18:19:27.761352 (+ 294us) client_negotiation.cc:240] Negotiation successful
0807 18:19:27.761835 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":312,"threads_started":1}
I20260807 18:19:27.765407 10666 heartbeater.cc:347] Connected to a master server at 127.17.110.62:46857
I20260807 18:19:27.765827 10666 heartbeater.cc:464] Registering TS with master...
I20260807 18:19:27.766691 10666 heartbeater.cc:511] Master 127.17.110.62:46857 requested a full tablet report, sending...
I20260807 18:19:27.768801 10173 ts_manager.cc:194] Registered new tserver with Master: de1af49afae24218968d33e1d584d8b6 (127.17.110.2:40261)
I20260807 18:19:27.770213 10173 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.2@KRBTEST.COM'} at 127.17.110.2:54069
WARNING: no policy specified for kudu/127.17.110.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
I20260807 18:19:27.822585 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.3:0
--local_ip_for_outbound_sockets=127.17.110.3
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:46857
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:33471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:19:28.150262 10458 heartbeater.cc:503] Master 127.17.110.62:46857 was elected leader, sending a full tablet report...
W20260807 18:19:28.195732 10673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:19:28.196066 10673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:19:28.196231 10673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:19:28.205274 10673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:19:28.205438 10673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:19:28.205634 10673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.3
I20260807 18:19:28.216261 10673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:33471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:46857
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.3
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:19:28.217751 10673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:19:28.219224 10673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:19:28.234864 10687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:28.234674 10685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:28.234753 10684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:19:28.235800 10673 server_base.cc:1034] running on GCE node
I20260807 18:19:28.236652 10673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:19:28.237869 10673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:19:28.239250 10673 hybrid_clock.cc:648] HybridClock initialized: now 1786126768239171 us; error 66 us; skew 500 ppm
I20260807 18:19:28.239675 10673 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:19:28 dist-test-slave-39l8 krb5kdc[10107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126768, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:19:28.243846 10673 init.cc:377] Logged in from keytab as kudu/127.17.110.3@KRBTEST.COM (short username kudu)
I20260807 18:19:28.245757 10673 webserver.cc:460] Webserver started at http://127.17.110.3:35963/ using document root <none> and password file <none>
I20260807 18:19:28.246752 10673 fs_manager.cc:359] Metadata directory not provided
I20260807 18:19:28.246973 10673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:19:28.247365 10673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:19:28.250995 10673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-2/data/instance:
uuid: "d5d72235b3884a1d80de3bdebb076551"
format_stamp: "Formatted at 2026-08-07 18:19:28 on dist-test-slave-39l8"
server_key: "89b6a7a56b365d5f6854d31a09dc04e5"
server_key_iv: "6aa5683809ffadb69054e253d6d2f117"
server_key_version: "encryptionkey@0"
I20260807 18:19:28.251995 10673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance:
uuid: "d5d72235b3884a1d80de3bdebb076551"
format_stamp: "Formatted at 2026-08-07 18:19:28 on dist-test-slave-39l8"
server_key: "89b6a7a56b365d5f6854d31a09dc04e5"
server_key_iv: "6aa5683809ffadb69054e253d6d2f117"
server_key_version: "encryptionkey@0"
I20260807 18:19:28.258932 10673 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260807 18:19:28.263391 10696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:28.265268 10673 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:19:28.265509 10673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-2/wal
uuid: "d5d72235b3884a1d80de3bdebb076551"
format_stamp: "Formatted at 2026-08-07 18:19:28 on dist-test-slave-39l8"
server_key: "89b6a7a56b365d5f6854d31a09dc04e5"
server_key_iv: "6aa5683809ffadb69054e253d6d2f117"
server_key_version: "encryptionkey@0"
I20260807 18:19:28.265786 10673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:19:28.290385 10673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:19:28.295535 10673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:19:28.295924 10673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:19:28.297137 10673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:19:28.298805 10673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:19:28.298898 10673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:28.299077 10673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:19:28.299136 10673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:28.338991 10673 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.3:44885
I20260807 18:19:28.339033 10860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.3:44885 every 8 connection(s)
I20260807 18:19:28.340391 10673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
I20260807 18:19:28.348935 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 10673
I20260807 18:19:28.349162 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance
I20260807 18:19:28.349673 17848 external_mini_cluster.cc:1428] Setting key a39c8d8f411c7775427ef93023f62ecf
Aug 07 18:19:28 dist-test-slave-39l8 krb5kdc[10107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126768, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:19:28.365906 10863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:28.343435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:44111 (local address 127.17.110.62:46857)
0807 18:19:28.344058 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:28.344063 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:28.345075 (+ 1012us) server_negotiation.cc:407] Connection header received
0807 18:19:28.347339 (+ 2264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:28.347345 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:28.347471 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:28.347612 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:28.349487 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:28.350549 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:28.352862 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:28.353490 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:28.353657 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:28.358007 (+ 4350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:28.358013 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:28.358017 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:19:28.358069 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:28.361725 (+ 3656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:28.362856 (+ 1131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:28.362864 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:28.362870 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:28.363037 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:28.363652 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:28.363658 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:28.363662 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:28.363992 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:28.364207 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:28.365144 (+ 937us) server_negotiation.cc:299] Negotiation successful
0807 18:19:28.365452 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":249,"threads_started":1}
I20260807 18:19:28.367374 10864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:28.344040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:46857 (local address 127.17.110.3:44111)
0807 18:19:28.344849 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:28.344893 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:28.346997 (+ 2104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:28.347905 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:28.347915 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:28.348546 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:19:28.349246 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:28.349269 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:28.350816 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:28.350822 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:28.352682 (+ 1860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:28.352697 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:28.353758 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:28.353767 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:28.354080 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:28.355028 (+ 948us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:19:28.355063 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:19:28.357695 (+ 2632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:19:28.361976 (+ 4281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:19:28.361988 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:19:28.362008 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:19:28.362585 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:19:28.363194 (+ 609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:19:28.363202 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:19:28.363207 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:19:28.363479 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:19:28.364173 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:19:28.364184 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:19:28.364627 (+ 443us) client_negotiation.cc:769] Sending connection context
0807 18:19:28.364977 (+ 350us) client_negotiation.cc:240] Negotiation successful
0807 18:19:28.365644 (+ 667us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":309,"threads_started":1}
I20260807 18:19:28.369417 10861 heartbeater.cc:347] Connected to a master server at 127.17.110.62:46857
I20260807 18:19:28.369817 10861 heartbeater.cc:464] Registering TS with master...
I20260807 18:19:28.370921 10861 heartbeater.cc:511] Master 127.17.110.62:46857 requested a full tablet report, sending...
I20260807 18:19:28.372793 10173 ts_manager.cc:194] Registered new tserver with Master: d5d72235b3884a1d80de3bdebb076551 (127.17.110.3:44885)
I20260807 18:19:28.373790 10173 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.3@KRBTEST.COM'} at 127.17.110.3:44111
I20260807 18:19:28.381215 17848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:19:28.403702 10863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:28.386584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33668 (local address 127.17.110.62:46857)
0807 18:19:28.386852 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:28.386858 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:28.387405 (+ 547us) server_negotiation.cc:407] Connection header received
0807 18:19:28.387567 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:28.387571 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:28.387658 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:28.387765 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:28.389385 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:28.390262 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:28.391792 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:28.392262 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:28.392383 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:28.395691 (+ 3308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:28.395697 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:28.395701 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:19:28.395753 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:28.399175 (+ 3422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:28.400288 (+ 1113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:28.400294 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:28.400296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:28.400475 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:28.401074 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:28.401080 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:28.401083 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:28.401618 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:28.401920 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:28.402583 (+ 663us) server_negotiation.cc:299] Negotiation successful
0807 18:19:28.403278 (+ 695us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 18:19:28.409505 10173 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33668:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:19:28.417043 10173 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:19:28.442610 10884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:28.432034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38040 (local address 127.17.110.3:44885)
0807 18:19:28.434236 (+ 2202us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:28.434249 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:28.434289 (+ 40us) server_negotiation.cc:407] Connection header received
0807 18:19:28.434406 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:28.434415 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:28.434760 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:28.434990 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:19:28.436109 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:28.437258 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:28.440188 (+ 2930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:28.441221 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:28.441398 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:28.441682 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:28.442043 (+ 361us) server_negotiation.cc:299] Negotiation successful
0807 18:19:28.442224 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1989,"thread_start_us":1818,"threads_started":1}
I20260807 18:19:28.442670 10882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:28.431052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:44885 (local address 127.0.0.1:38040)
0807 18:19:28.433309 (+ 2257us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:28.433358 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:28.433542 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:28.435360 (+ 1818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:28.435367 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:28.435413 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:19:28.435870 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:28.435885 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:28.437460 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:28.437468 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:28.439999 (+ 2531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:28.440012 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:28.441461 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:28.441468 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:28.441686 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:28.441757 (+ 71us) client_negotiation.cc:769] Sending connection context
0807 18:19:28.441975 (+ 218us) client_negotiation.cc:240] Negotiation successful
0807 18:19:28.442223 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1876,"thread_start_us":3342,"threads_started":1}
I20260807 18:19:28.446452 10883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:28.431270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42546 (local address 127.17.110.2:40261)
0807 18:19:28.432565 (+ 1295us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:28.432581 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:28.434991 (+ 2410us) server_negotiation.cc:407] Connection header received
0807 18:19:28.435396 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:28.435407 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:28.435695 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:28.435904 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:19:28.437258 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:28.439732 (+ 2474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:28.443517 (+ 3785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:28.445255 (+ 1738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:28.445430 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:28.445666 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:28.445934 (+ 268us) server_negotiation.cc:299] Negotiation successful
0807 18:19:28.446112 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1035,"thread_start_us":911,"threads_started":1}
I20260807 18:19:28.446483 10880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:28.431096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:40261 (local address 127.0.0.1:42546)
0807 18:19:28.434828 (+ 3732us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:28.434864 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:28.435147 (+ 283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:28.436323 (+ 1176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:28.436331 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:28.436376 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:19:28.436976 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:28.436995 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:28.439916 (+ 2921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:28.439927 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:28.442503 (+ 2576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:28.442533 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:28.445489 (+ 2956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:28.445497 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:28.445697 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:28.445746 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 18:19:28.445880 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 18:19:28.446037 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3511,"mutex_wait_us":1,"thread_start_us":1666,"threads_started":1}
I20260807 18:19:28.447870 10885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:28.435339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49650 (local address 127.17.110.1:43871)
0807 18:19:28.436091 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:28.436099 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:28.437914 (+ 1815us) server_negotiation.cc:407] Connection header received
0807 18:19:28.438168 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:28.438180 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:28.438485 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:28.438772 (+ 287us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:19:28.439584 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:28.441528 (+ 1944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:28.444433 (+ 2905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:28.446246 (+ 1813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:28.446415 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:28.446669 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:28.447328 (+ 659us) server_negotiation.cc:299] Negotiation successful
0807 18:19:28.447517 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":633,"threads_started":1}
I20260807 18:19:28.448022 10765 tablet_service.cc:1467] Processing CreateTablet for tablet 495256131a02424a89a3458ac459a3b8 (DEFAULT_TABLE table=test-table [id=3841155c8d5d4909aea43ceeb12e27af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:19:28.447822 10886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:28.435784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:43871 (local address 127.0.0.1:49650)
0807 18:19:28.437753 (+ 1969us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:28.437778 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:28.437957 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:28.438787 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:28.438795 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:28.438829 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:19:28.439357 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:28.439373 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:28.441741 (+ 2368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:28.441752 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:28.444082 (+ 2330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:28.444098 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:28.446870 (+ 2772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:28.446878 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:28.447090 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:28.447129 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 18:19:28.447251 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 18:19:28.447414 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1819,"thread_start_us":1619,"threads_started":1}
I20260807 18:19:28.450063 10765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 495256131a02424a89a3458ac459a3b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:19:28.451377 10574 tablet_service.cc:1467] Processing CreateTablet for tablet 495256131a02424a89a3458ac459a3b8 (DEFAULT_TABLE table=test-table [id=3841155c8d5d4909aea43ceeb12e27af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:19:28.453228 10574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 495256131a02424a89a3458ac459a3b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:19:28.453512 10369 tablet_service.cc:1467] Processing CreateTablet for tablet 495256131a02424a89a3458ac459a3b8 (DEFAULT_TABLE table=test-table [id=3841155c8d5d4909aea43ceeb12e27af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:19:28.455096 10369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 495256131a02424a89a3458ac459a3b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:19:28.468192 10888 tablet_bootstrap.cc:492] T 495256131a02424a89a3458ac459a3b8 P d5d72235b3884a1d80de3bdebb076551: Bootstrap starting.
I20260807 18:19:28.470643 10889 tablet_bootstrap.cc:492] T 495256131a02424a89a3458ac459a3b8 P ef37ccc01ac7442d9a80b178d5251e68: Bootstrap starting.
I20260807 18:19:28.472043 10890 tablet_bootstrap.cc:492] T 495256131a02424a89a3458ac459a3b8 P de1af49afae24218968d33e1d584d8b6: Bootstrap starting.
I20260807 18:19:28.473739 10888 tablet_bootstrap.cc:654] T 495256131a02424a89a3458ac459a3b8 P d5d72235b3884a1d80de3bdebb076551: Neither blocks nor log segments found. Creating new log.
I20260807 18:19:28.475137 10889 tablet_bootstrap.cc:654] T 495256131a02424a89a3458ac459a3b8 P ef37ccc01ac7442d9a80b178d5251e68: Neither blocks nor log segments found. Creating new log.
I20260807 18:19:28.475315 10888 log.cc:824] T 495256131a02424a89a3458ac459a3b8 P d5d72235b3884a1d80de3bdebb076551: Log is configured to *not* fsync() on all Append() calls
I20260807 18:19:28.475864 10890 tablet_bootstrap.cc:654] T 495256131a02424a89a3458ac459a3b8 P de1af49afae24218968d33e1d584d8b6: Neither blocks nor log segments found. Creating new log.
I20260807 18:19:28.476491 10889 log.cc:824] T 495256131a02424a89a3458ac459a3b8 P ef37ccc01ac7442d9a80b178d5251e68: Log is configured to *not* fsync() on all Append() calls
I20260807 18:19:28.477021 10890 log.cc:824] T 495256131a02424a89a3458ac459a3b8 P de1af49afae24218968d33e1d584d8b6: Log is configured to *not* fsync() on all Append() calls
I20260807 18:19:28.479348 10888 tablet_bootstrap.cc:492] T 495256131a02424a89a3458ac459a3b8 P d5d72235b3884a1d80de3bdebb076551: No bootstrap required, opened a new log
I20260807 18:19:28.479744 10888 ts_tablet_manager.cc:1397] T 495256131a02424a89a3458ac459a3b8 P d5d72235b3884a1d80de3bdebb076551: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260807 18:19:28.480033 10890 tablet_bootstrap.cc:492] T 495256131a02424a89a3458ac459a3b8 P de1af49afae24218968d33e1d584d8b6: No bootstrap required, opened a new log
I20260807 18:19:28.480055 10889 tablet_bootstrap.cc:492] T 495256131a02424a89a3458ac459a3b8 P ef37ccc01ac7442d9a80b178d5251e68: No bootstrap required, opened a new log
I20260807 18:19:28.480337 10890 ts_tablet_manager.cc:1397] T 495256131a02424a89a3458ac459a3b8 P de1af49afae24218968d33e1d584d8b6: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 18:19:28.480335 10889 ts_tablet_manager.cc:1397] T 495256131a02424a89a3458ac459a3b8 P ef37ccc01ac7442d9a80b178d5251e68: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260807 18:19:28.488771 10890 raft_consensus.cc:348] T 495256131a02424a89a3458ac459a3b8 P de1af49afae24218968d33e1d584d8b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de1af49afae24218968d33e1d584d8b6" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40261 } } peers { permanent_uuid: "d5d72235b3884a1d80de3bdebb076551" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44885 } } peers { permanent_uuid: "ef37ccc01ac7442d9a80b178d5251e68" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 43871 } }
I20260807 18:19:28.489243 10890 raft_consensus.cc:374] T 495256131a02424a89a3458ac459a3b8 P de1af49afae24218968d33e1d584d8b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:19:28.489346 10890 raft_consensus.cc:729] T 495256131a02424a89a3458ac459a3b8 P de1af49afae24218968d33e1d584d8b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de1af49afae24218968d33e1d584d8b6, State: Initialized, Role: FOLLOWER
I20260807 18:19:28.490110 10890 consensus_queue.cc:260] T 495256131a02424a89a3458ac459a3b8 P de1af49afae24218968d33e1d584d8b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de1af49afae24218968d33e1d584d8b6" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40261 } } peers { permanent_uuid: "d5d72235b3884a1d80de3bdebb076551" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44885 } } peers { permanent_uuid: "ef37ccc01ac7442d9a80b178d5251e68" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 43871 } }
I20260807 18:19:28.491418 10666 heartbeater.cc:503] Master 127.17.110.62:46857 was elected leader, sending a full tablet report...
I20260807 18:19:28.491477 10889 raft_consensus.cc:348] T 495256131a02424a89a3458ac459a3b8 P ef37ccc01ac7442d9a80b178d5251e68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de1af49afae24218968d33e1d584d8b6" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40261 } } peers { permanent_uuid: "d5d72235b3884a1d80de3bdebb076551" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44885 } } peers { permanent_uuid: "ef37ccc01ac7442d9a80b178d5251e68" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 43871 } }
I20260807 18:19:28.491477 10888 raft_consensus.cc:348] T 495256131a02424a89a3458ac459a3b8 P d5d72235b3884a1d80de3bdebb076551 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de1af49afae24218968d33e1d584d8b6" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40261 } } peers { permanent_uuid: "d5d72235b3884a1d80de3bdebb076551" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44885 } } peers { permanent_uuid: "ef37ccc01ac7442d9a80b178d5251e68" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 43871 } }
I20260807 18:19:28.491968 10889 raft_consensus.cc:374] T 495256131a02424a89a3458ac459a3b8 P ef37ccc01ac7442d9a80b178d5251e68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:19:28.491972 10888 raft_consensus.cc:374] T 495256131a02424a89a3458ac459a3b8 P d5d72235b3884a1d80de3bdebb076551 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:19:28.492096 10888 raft_consensus.cc:729] T 495256131a02424a89a3458ac459a3b8 P d5d72235b3884a1d80de3bdebb076551 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5d72235b3884a1d80de3bdebb076551, State: Initialized, Role: FOLLOWER
I20260807 18:19:28.492096 10889 raft_consensus.cc:729] T 495256131a02424a89a3458ac459a3b8 P ef37ccc01ac7442d9a80b178d5251e68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef37ccc01ac7442d9a80b178d5251e68, State: Initialized, Role: FOLLOWER
I20260807 18:19:28.492238 10890 ts_tablet_manager.cc:1428] T 495256131a02424a89a3458ac459a3b8 P de1af49afae24218968d33e1d584d8b6: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260807 18:19:28.492910 10888 consensus_queue.cc:260] T 495256131a02424a89a3458ac459a3b8 P d5d72235b3884a1d80de3bdebb076551 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de1af49afae24218968d33e1d584d8b6" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40261 } } peers { permanent_uuid: "d5d72235b3884a1d80de3bdebb076551" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44885 } } peers { permanent_uuid: "ef37ccc01ac7442d9a80b178d5251e68" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 43871 } }
I20260807 18:19:28.492911 10889 consensus_queue.cc:260] T 495256131a02424a89a3458ac459a3b8 P ef37ccc01ac7442d9a80b178d5251e68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de1af49afae24218968d33e1d584d8b6" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40261 } } peers { permanent_uuid: "d5d72235b3884a1d80de3bdebb076551" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44885 } } peers { permanent_uuid: "ef37ccc01ac7442d9a80b178d5251e68" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 43871 } }
I20260807 18:19:28.495051 10888 ts_tablet_manager.cc:1428] T 495256131a02424a89a3458ac459a3b8 P d5d72235b3884a1d80de3bdebb076551: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260807 18:19:28.495401 10889 ts_tablet_manager.cc:1428] T 495256131a02424a89a3458ac459a3b8 P ef37ccc01ac7442d9a80b178d5251e68: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260807 18:19:28.495613 10861 heartbeater.cc:503] Master 127.17.110.62:46857 was elected leader, sending a full tablet report...
I20260807 18:19:28.580948 10897 raft_consensus.cc:482] T 495256131a02424a89a3458ac459a3b8 P de1af49afae24218968d33e1d584d8b6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:19:28.581328 10897 raft_consensus.cc:504] T 495256131a02424a89a3458ac459a3b8 P de1af49afae24218968d33e1d584d8b6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de1af49afae24218968d33e1d584d8b6" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40261 } } peers { permanent_uuid: "d5d72235b3884a1d80de3bdebb076551" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44885 } } peers { permanent_uuid: "ef37ccc01ac7442d9a80b178d5251e68" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 43871 } }
I20260807 18:19:28.583343 10897 leader_election.cc:288] T 495256131a02424a89a3458ac459a3b8 P de1af49afae24218968d33e1d584d8b6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5d72235b3884a1d80de3bdebb076551 (127.17.110.3:44885), ef37ccc01ac7442d9a80b178d5251e68 (127.17.110.1:43871)
W20260807 18:19:28.592164 10862 tablet.cc:2369] T 495256131a02424a89a3458ac459a3b8 P d5d72235b3884a1d80de3bdebb076551: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:19:28.592507 10884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:28.583850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:34521 (local address 127.17.110.3:44885)
0807 18:19:28.584160 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:28.584168 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:28.584713 (+ 545us) server_negotiation.cc:407] Connection header received
0807 18:19:28.584992 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:28.585001 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:28.585092 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:28.585207 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:19:28.585928 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:28.587044 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:28.590150 (+ 3106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:28.591235 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:28.591436 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:28.591527 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:28.591997 (+ 470us) server_negotiation.cc:299] Negotiation successful
0807 18:19:28.592195 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:19:28.592648 10901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:28.583676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:44885 (local address 127.17.110.2:34521)
0807 18:19:28.584511 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:28.584584 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:28.584805 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:28.585251 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:28.585259 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:28.585300 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:19:28.585756 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:28.585766 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:28.587197 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:28.587202 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:28.589913 (+ 2711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:28.589936 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:28.591505 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:28.591512 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:28.591700 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:28.591727 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 18:19:28.591921 (+ 194us) client_negotiation.cc:240] Negotiation successful
0807 18:19:28.592187 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"mutex_wait_us":50,"thread_start_us":397,"threads_started":1}
I20260807 18:19:28.593286 10885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:28.584792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:37053 (local address 127.17.110.1:43871)
0807 18:19:28.584981 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:28.584987 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:28.585476 (+ 489us) server_negotiation.cc:407] Connection header received
0807 18:19:28.585704 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:28.585708 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:28.585778 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:28.585915 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:19:28.586475 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:28.587491 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:28.590310 (+ 2819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:28.591482 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:28.591708 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:28.591826 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:28.592300 (+ 474us) server_negotiation.cc:299] Negotiation successful
0807 18:19:28.592969 (+ 669us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 18:19:28.593434 10902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:28.584539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:43871 (local address 127.17.110.2:37053)
0807 18:19:28.585282 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:28.585301 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:28.585473 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:28.585946 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:28.585950 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:28.585972 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:19:28.586303 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:28.586310 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:28.587641 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:28.587647 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:28.590089 (+ 2442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:28.590106 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:28.591764 (+ 1658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:28.591772 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:28.592071 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:28.592111 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 18:19:28.592838 (+ 727us) client_negotiation.cc:240] Negotiation successful
0807 18:19:28.592999 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"mutex_wait_us":1,"thread_start_us":450,"threads_started":1}
I20260807 18:19:28.593824 10794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "495256131a02424a89a3458ac459a3b8" candidate_uuid: "de1af49afae24218968d33e1d584d8b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5d72235b3884a1d80de3bdebb076551" is_pre_election: true
I20260807 18:19:28.594396 10794 raft_consensus.cc:2454] T 495256131a02424a89a3458ac459a3b8 P d5d72235b3884a1d80de3bdebb076551 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de1af49afae24218968d33e1d584d8b6 in term 0.
I20260807 18:19:28.594641 10397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "495256131a02424a89a3458ac459a3b8" candidate_uuid: "de1af49afae24218968d33e1d584d8b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef37ccc01ac7442d9a80b178d5251e68" is_pre_election: true
I20260807 18:19:28.595199 10500 leader_election.cc:302] T 495256131a02424a89a3458ac459a3b8 P de1af49afae24218968d33e1d584d8b6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d5d72235b3884a1d80de3bdebb076551, de1af49afae24218968d33e1d584d8b6; no voters:
I20260807 18:19:28.595285 10397 raft_consensus.cc:2454] T 495256131a02424a89a3458ac459a3b8 P ef37ccc01ac7442d9a80b178d5251e68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de1af49afae24218968d33e1d584d8b6 in term 0.
I20260807 18:19:28.595646 10897 raft_consensus.cc:2781] T 495256131a02424a89a3458ac459a3b8 P de1af49afae24218968d33e1d584d8b6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:19:28.595808 10897 raft_consensus.cc:482] T 495256131a02424a89a3458ac459a3b8 P de1af49afae24218968d33e1d584d8b6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:19:28.595932 10897 raft_consensus.cc:3037] T 495256131a02424a89a3458ac459a3b8 P de1af49afae24218968d33e1d584d8b6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:19:28.598281 10897 raft_consensus.cc:504] T 495256131a02424a89a3458ac459a3b8 P de1af49afae24218968d33e1d584d8b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de1af49afae24218968d33e1d584d8b6" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40261 } } peers { permanent_uuid: "d5d72235b3884a1d80de3bdebb076551" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44885 } } peers { permanent_uuid: "ef37ccc01ac7442d9a80b178d5251e68" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 43871 } }
I20260807 18:19:28.599104 10897 leader_election.cc:288] T 495256131a02424a89a3458ac459a3b8 P de1af49afae24218968d33e1d584d8b6 [CANDIDATE]: Term 1 election: Requested vote from peers d5d72235b3884a1d80de3bdebb076551 (127.17.110.3:44885), ef37ccc01ac7442d9a80b178d5251e68 (127.17.110.1:43871)
I20260807 18:19:28.599716 10794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "495256131a02424a89a3458ac459a3b8" candidate_uuid: "de1af49afae24218968d33e1d584d8b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5d72235b3884a1d80de3bdebb076551"
I20260807 18:19:28.599764 10397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "495256131a02424a89a3458ac459a3b8" candidate_uuid: "de1af49afae24218968d33e1d584d8b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef37ccc01ac7442d9a80b178d5251e68"
I20260807 18:19:28.599946 10794 raft_consensus.cc:3037] T 495256131a02424a89a3458ac459a3b8 P d5d72235b3884a1d80de3bdebb076551 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:19:28.600059 10397 raft_consensus.cc:3037] T 495256131a02424a89a3458ac459a3b8 P ef37ccc01ac7442d9a80b178d5251e68 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:19:28.602435 10794 raft_consensus.cc:2454] T 495256131a02424a89a3458ac459a3b8 P d5d72235b3884a1d80de3bdebb076551 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de1af49afae24218968d33e1d584d8b6 in term 1.
I20260807 18:19:28.602810 10397 raft_consensus.cc:2454] T 495256131a02424a89a3458ac459a3b8 P ef37ccc01ac7442d9a80b178d5251e68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de1af49afae24218968d33e1d584d8b6 in term 1.
I20260807 18:19:28.603026 10500 leader_election.cc:302] T 495256131a02424a89a3458ac459a3b8 P de1af49afae24218968d33e1d584d8b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d5d72235b3884a1d80de3bdebb076551, de1af49afae24218968d33e1d584d8b6; no voters:
I20260807 18:19:28.603585 10897 raft_consensus.cc:2781] T 495256131a02424a89a3458ac459a3b8 P de1af49afae24218968d33e1d584d8b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:19:28.604178 10897 raft_consensus.cc:686] T 495256131a02424a89a3458ac459a3b8 P de1af49afae24218968d33e1d584d8b6 [term 1 LEADER]: Becoming Leader. State: Replica: de1af49afae24218968d33e1d584d8b6, State: Running, Role: LEADER
I20260807 18:19:28.604781 10897 consensus_queue.cc:237] T 495256131a02424a89a3458ac459a3b8 P de1af49afae24218968d33e1d584d8b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de1af49afae24218968d33e1d584d8b6" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40261 } } peers { permanent_uuid: "d5d72235b3884a1d80de3bdebb076551" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44885 } } peers { permanent_uuid: "ef37ccc01ac7442d9a80b178d5251e68" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 43871 } }
I20260807 18:19:28.610944 10170 catalog_manager.cc:5495] T 495256131a02424a89a3458ac459a3b8 P de1af49afae24218968d33e1d584d8b6 reported cstate change: term changed from 0 to 1, leader changed from <none> to de1af49afae24218968d33e1d584d8b6 (127.17.110.2). New cstate: current_term: 1 leader_uuid: "de1af49afae24218968d33e1d584d8b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de1af49afae24218968d33e1d584d8b6" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5d72235b3884a1d80de3bdebb076551" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef37ccc01ac7442d9a80b178d5251e68" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 43871 } health_report { overall_health: UNKNOWN } } }
W20260807 18:19:28.613427 10459 tablet.cc:2369] T 495256131a02424a89a3458ac459a3b8 P ef37ccc01ac7442d9a80b178d5251e68: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:19:28.642145 10883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:28.636102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42554 (local address 127.17.110.2:40261)
0807 18:19:28.636309 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:28.636314 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:28.636358 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:19:28.636588 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:28.636592 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:28.636689 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:28.636834 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:19:28.637316 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:28.638033 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:28.639860 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:28.640267 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:28.640368 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:28.640996 (+ 628us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:19:28.641584 (+ 588us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:19:28.641700 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:28.641847 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 18:19:28.641926 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:19:28.651873 10397 raft_consensus.cc:1261] T 495256131a02424a89a3458ac459a3b8 P ef37ccc01ac7442d9a80b178d5251e68 [term 1 FOLLOWER]: Refusing update from remote peer de1af49afae24218968d33e1d584d8b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:19:28.652108 10794 raft_consensus.cc:1261] T 495256131a02424a89a3458ac459a3b8 P d5d72235b3884a1d80de3bdebb076551 [term 1 FOLLOWER]: Refusing update from remote peer de1af49afae24218968d33e1d584d8b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:19:28.653173 10903 consensus_queue.cc:1035] T 495256131a02424a89a3458ac459a3b8 P de1af49afae24218968d33e1d584d8b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef37ccc01ac7442d9a80b178d5251e68" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 43871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:19:28.653667 10897 consensus_queue.cc:1035] T 495256131a02424a89a3458ac459a3b8 P de1af49afae24218968d33e1d584d8b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5d72235b3884a1d80de3bdebb076551" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:19:28.689191 10170 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:19:28.689692 10170 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:19:28.694553 10170 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 00a893209cf644609c81e01b2f7dbad3: Sending DeleteTablet for 3 replicas of tablet 495256131a02424a89a3458ac459a3b8
I20260807 18:19:28.696273 10765 tablet_service.cc:1514] Processing DeleteTablet for tablet 495256131a02424a89a3458ac459a3b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:19:28 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:38040
I20260807 18:19:28.696216 10574 tablet_service.cc:1514] Processing DeleteTablet for tablet 495256131a02424a89a3458ac459a3b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:19:28 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:42546
I20260807 18:19:28.696694 10369 tablet_service.cc:1514] Processing DeleteTablet for tablet 495256131a02424a89a3458ac459a3b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:19:28 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:49650
I20260807 18:19:28.699345 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 10271
I20260807 18:19:28.708665 10924 tablet_replica.cc:330] T 495256131a02424a89a3458ac459a3b8 P d5d72235b3884a1d80de3bdebb076551: stopping tablet replica
I20260807 18:19:28.709368 10924 raft_consensus.cc:2229] T 495256131a02424a89a3458ac459a3b8 P d5d72235b3884a1d80de3bdebb076551 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:19:28.710189 10924 raft_consensus.cc:2258] T 495256131a02424a89a3458ac459a3b8 P d5d72235b3884a1d80de3bdebb076551 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:19:28.716617 10924 ts_tablet_manager.cc:1905] T 495256131a02424a89a3458ac459a3b8 P d5d72235b3884a1d80de3bdebb076551: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:19:28.724658 10925 tablet_replica.cc:330] T 495256131a02424a89a3458ac459a3b8 P de1af49afae24218968d33e1d584d8b6: stopping tablet replica
I20260807 18:19:28.725440 10926 tablet_replica.cc:330] T 495256131a02424a89a3458ac459a3b8 P ef37ccc01ac7442d9a80b178d5251e68: stopping tablet replica
I20260807 18:19:28.726102 10925 raft_consensus.cc:2229] T 495256131a02424a89a3458ac459a3b8 P de1af49afae24218968d33e1d584d8b6 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:19:28.726999 10925 raft_consensus.cc:2258] T 495256131a02424a89a3458ac459a3b8 P de1af49afae24218968d33e1d584d8b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:19:28.728483 10926 raft_consensus.cc:2229] T 495256131a02424a89a3458ac459a3b8 P ef37ccc01ac7442d9a80b178d5251e68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:19:28.728921 10924 ts_tablet_manager.cc:1918] T 495256131a02424a89a3458ac459a3b8 P d5d72235b3884a1d80de3bdebb076551: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:19:28.729153 10924 log.cc:1196] T 495256131a02424a89a3458ac459a3b8 P d5d72235b3884a1d80de3bdebb076551: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-2/wal/wals/495256131a02424a89a3458ac459a3b8
I20260807 18:19:28.729153 10926 raft_consensus.cc:2258] T 495256131a02424a89a3458ac459a3b8 P ef37ccc01ac7442d9a80b178d5251e68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:19:28.730075 10924 ts_tablet_manager.cc:1939] T 495256131a02424a89a3458ac459a3b8 P d5d72235b3884a1d80de3bdebb076551: Deleting consensus metadata
I20260807 18:19:28.730317 10925 ts_tablet_manager.cc:1905] T 495256131a02424a89a3458ac459a3b8 P de1af49afae24218968d33e1d584d8b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:19:28.732375 10155 catalog_manager.cc:4841] TS d5d72235b3884a1d80de3bdebb076551 (127.17.110.3:44885): tablet 495256131a02424a89a3458ac459a3b8 (table test-table [id=3841155c8d5d4909aea43ceeb12e27af]) successfully deleted
Aug 07 18:19:28 dist-test-slave-39l8 krb5kdc[10107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.1@KRBTEST.COM
I20260807 18:19:28.734929 10926 ts_tablet_manager.cc:1905] T 495256131a02424a89a3458ac459a3b8 P ef37ccc01ac7442d9a80b178d5251e68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:19:28.741199 10925 ts_tablet_manager.cc:1918] T 495256131a02424a89a3458ac459a3b8 P de1af49afae24218968d33e1d584d8b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:19:28.741526 10925 log.cc:1196] T 495256131a02424a89a3458ac459a3b8 P de1af49afae24218968d33e1d584d8b6: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126429568977-17848-0/minicluster-data/ts-1/wal/wals/495256131a02424a89a3458ac459a3b8
I20260807 18:19:28.742205 10925 ts_tablet_manager.cc:1939] T 495256131a02424a89a3458ac459a3b8 P de1af49afae24218968d33e1d584d8b6: Deleting consensus metadata
I20260807 18:19:28.744596 10157 catalog_manager.cc:4841] TS de1af49afae24218968d33e1d584d8b6 (127.17.110.2:40261): tablet 495256131a02424a89a3458ac459a3b8 (table test-table [id=3841155c8d5d4909aea43ceeb12e27af]) successfully deleted
I20260807 18:19:28.745183 10885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:28.701727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49666 (local address 127.17.110.1:43871)
0807 18:19:28.702003 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:28.702014 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:28.716193 (+ 14179us) server_negotiation.cc:407] Connection header received
0807 18:19:28.716949 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:28.716956 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:28.717073 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:28.717181 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:28.719626 (+ 2445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:28.721452 (+ 1826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:28.726079 (+ 4627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:28.726916 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:28.727085 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:28.734997 (+ 7912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:28.735018 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:28.735040 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:19:28.735106 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:28.739686 (+ 4580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:28.740925 (+ 1239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:28.740938 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:28.740942 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:28.741175 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:28.742619 (+ 1444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:28.742627 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:28.742631 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:28.743210 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:28.743701 (+ 491us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:28.744444 (+ 743us) server_negotiation.cc:299] Negotiation successful
0807 18:19:28.744748 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 18:19:28.746796 10450 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:19:28.874823 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 10271
I20260807 18:19:28.899806 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 10469
W20260807 18:19:28.899916 10155 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.110.1:43871 (error 108)
I20260807 18:19:28.902309 10886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:28.901857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:43871 (local address 127.0.0.1:49680)
0807 18:19:28.902026 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:28.902143 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:43871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260807 18:19:28.902670 10155 catalog_manager.cc:4584] TS ef37ccc01ac7442d9a80b178d5251e68 (127.17.110.1:43871): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 495256131a02424a89a3458ac459a3b8: Network error: Client connection negotiation failed: client connection to 127.17.110.1:43871: connect: Connection refused (error 111)
Aug 07 18:19:28 dist-test-slave-39l8 krb5kdc[10107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.2@KRBTEST.COM
I20260807 18:19:28.919912 10883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:28.900686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42562 (local address 127.17.110.2:40261)
0807 18:19:28.900865 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:28.900871 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:28.901138 (+ 267us) server_negotiation.cc:407] Connection header received
0807 18:19:28.901229 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:28.901235 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:28.901336 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:28.901449 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:28.903367 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:28.904395 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:28.906359 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:28.907020 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:28.907135 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:28.912781 (+ 5646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:28.912802 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:28.912820 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:19:28.912875 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:28.916504 (+ 3629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:28.917397 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:28.917406 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:28.917409 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:28.917577 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:28.918107 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:28.918119 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:28.918122 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:28.918534 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:28.918986 (+ 452us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:28.919238 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 18:19:28.919491 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 18:19:28.920710 10657 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:19:28.956590 10886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:28.956065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:43871 (local address 127.0.0.1:49686)
0807 18:19:28.956311 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:28.956420 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:43871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 18:19:28.957929 10886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:28.957477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:43871 (local address 127.0.0.1:49702)
0807 18:19:28.957687 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:28.957770 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:43871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 18:19:29.018558 10886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:29.018045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:43871 (local address 127.0.0.1:49710)
0807 18:19:29.018285 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:29.018373 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:43871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 18:19:29.019694 10886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:29.019328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:43871 (local address 127.0.0.1:49724)
0807 18:19:29.019487 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:29.019568 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:43871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 18:19:29.040073 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 10469
I20260807 18:19:29.068329 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 10673
Aug 07 18:19:29 dist-test-slave-39l8 krb5kdc[10107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.3@KRBTEST.COM
I20260807 18:19:29.087275 10884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:29.069365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38042 (local address 127.17.110.3:44885)
0807 18:19:29.069637 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:29.069644 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:29.069671 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:19:29.069803 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:29.069809 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:29.069891 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:29.070019 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:29.071836 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:29.072691 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:29.074241 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:29.074872 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:29.075051 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:29.080758 (+ 5707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:29.080779 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:29.080796 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:19:29.080843 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:29.084024 (+ 3181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:29.084949 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:29.084958 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:29.084961 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:29.085169 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:29.085623 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:29.085627 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:29.085629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:29.086016 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:29.086324 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:29.086571 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 18:19:29.086955 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 18:19:29.088132 10854 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:19:29.100811 10886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:29.100081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:43871 (local address 127.0.0.1:49740)
0807 18:19:29.100380 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:29.100526 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:43871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260807 18:19:29.102273 10886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:29.101687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:43871 (local address 127.0.0.1:49750)
0807 18:19:29.101968 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:29.102084 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:43871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260807 18:19:29.181777 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 10673
I20260807 18:19:29.206658 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 10127
I20260807 18:19:29.208112 10219 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:19:29.322398 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 10127
2026-08-07T18:19:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (5155 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T18:19:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:19:29Z Disabled control of system clock
I20260807 18:19:29.430537 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:42519
--webserver_interface=127.17.110.62
--webserver_port=0
--builtin_ntp_servers=127.17.110.20:45817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:42519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:19:29.807518 10942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:19:29.807849 10942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:19:29.808014 10942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:19:29.815855 10942 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:19:29.815989 10942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:19:29.816068 10942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:19:29.816166 10942 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:19:29.816262 10942 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:19:29.826216 10942 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:45817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:42519
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:42519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:19:29.827736 10942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:19:29.829929 10942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:19:29.843583 10949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:29.843528 10954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:29.843528 10950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:19:29.844447 10942 server_base.cc:1034] running on GCE node
I20260807 18:19:29.845400 10942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:19:29.847067 10942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:19:29.848368 10942 hybrid_clock.cc:648] HybridClock initialized: now 1786126769848300 us; error 72 us; skew 500 ppm
I20260807 18:19:29.848793 10942 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:19:29.851626 10942 webserver.cc:460] Webserver started at http://127.17.110.62:39677/ using document root <none> and password file <none>
I20260807 18:19:29.852454 10942 fs_manager.cc:359] Metadata directory not provided
I20260807 18:19:29.852610 10942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:19:29.852968 10942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:19:29.856384 10942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "981d367211b441f3b7d734291a9a8fde"
format_stamp: "Formatted at 2026-08-07 18:19:29 on dist-test-slave-39l8"
server_key: "e27bb235868cc297eb85635e855f9926"
server_key_iv: "66508e9dc27446b8eb5eec6fda0eefb8"
server_key_version: "encryptionkey@0"
I20260807 18:19:29.857379 10942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "981d367211b441f3b7d734291a9a8fde"
format_stamp: "Formatted at 2026-08-07 18:19:29 on dist-test-slave-39l8"
server_key: "e27bb235868cc297eb85635e855f9926"
server_key_iv: "66508e9dc27446b8eb5eec6fda0eefb8"
server_key_version: "encryptionkey@0"
I20260807 18:19:29.863380 10942 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 18:19:29.867542 10961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:29.869091 10942 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:19:29.869359 10942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "981d367211b441f3b7d734291a9a8fde"
format_stamp: "Formatted at 2026-08-07 18:19:29 on dist-test-slave-39l8"
server_key: "e27bb235868cc297eb85635e855f9926"
server_key_iv: "66508e9dc27446b8eb5eec6fda0eefb8"
server_key_version: "encryptionkey@0"
I20260807 18:19:29.869640 10942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:19:29.912820 10942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:19:29.913869 10942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:19:29.914158 10942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:19:29.935796 10942 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:42519
I20260807 18:19:29.935827 11038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:42519 every 8 connection(s)
I20260807 18:19:29.937358 10942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:19:29.941701 11040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:19:29.946579 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 10942
I20260807 18:19:29.946822 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:19:29.947428 17848 external_mini_cluster.cc:1428] Setting key c851981faca6e8bdc1af4974af75b30c
I20260807 18:19:29.955041 11040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 981d367211b441f3b7d734291a9a8fde: Bootstrap starting.
I20260807 18:19:29.958508 11040 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 981d367211b441f3b7d734291a9a8fde: Neither blocks nor log segments found. Creating new log.
I20260807 18:19:29.959820 11040 log.cc:824] T 00000000000000000000000000000000 P 981d367211b441f3b7d734291a9a8fde: Log is configured to *not* fsync() on all Append() calls
I20260807 18:19:29.963200 11044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:29.949743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58232 (local address 127.17.110.62:42519)
0807 18:19:29.951219 (+ 1476us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:29.951241 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:29.951302 (+ 61us) server_negotiation.cc:407] Connection header received
0807 18:19:29.953462 (+ 2160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:29.953491 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:29.954027 (+ 536us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:29.954453 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:29.955098 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:29.956519 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:29.958003 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:29.959031 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:29.959271 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:29.960635 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:29.960646 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:29.960663 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:19:29.961271 (+ 608us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:29.961380 (+ 109us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:19:29.961404 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:29.961544 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:29.961808 (+ 264us) server_negotiation.cc:299] Negotiation successful
0807 18:19:29.962147 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":831,"thread_start_us":1457,"threads_started":1}
I20260807 18:19:29.963445 11040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 981d367211b441f3b7d734291a9a8fde: No bootstrap required, opened a new log
I20260807 18:19:29.972187 11040 raft_consensus.cc:348] T 00000000000000000000000000000000 P 981d367211b441f3b7d734291a9a8fde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "981d367211b441f3b7d734291a9a8fde" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 42519 } }
I20260807 18:19:29.972682 11040 raft_consensus.cc:374] T 00000000000000000000000000000000 P 981d367211b441f3b7d734291a9a8fde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:19:29.972802 11040 raft_consensus.cc:729] T 00000000000000000000000000000000 P 981d367211b441f3b7d734291a9a8fde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 981d367211b441f3b7d734291a9a8fde, State: Initialized, Role: FOLLOWER
I20260807 18:19:29.973604 11040 consensus_queue.cc:260] T 00000000000000000000000000000000 P 981d367211b441f3b7d734291a9a8fde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "981d367211b441f3b7d734291a9a8fde" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 42519 } }
I20260807 18:19:29.974035 11040 raft_consensus.cc:388] T 00000000000000000000000000000000 P 981d367211b441f3b7d734291a9a8fde [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:19:29.974211 11040 raft_consensus.cc:482] T 00000000000000000000000000000000 P 981d367211b441f3b7d734291a9a8fde [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:19:29.974427 11040 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 981d367211b441f3b7d734291a9a8fde [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:19:29.976980 11040 raft_consensus.cc:504] T 00000000000000000000000000000000 P 981d367211b441f3b7d734291a9a8fde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "981d367211b441f3b7d734291a9a8fde" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 42519 } }
I20260807 18:19:29.977604 11040 leader_election.cc:302] T 00000000000000000000000000000000 P 981d367211b441f3b7d734291a9a8fde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 981d367211b441f3b7d734291a9a8fde; no voters:
I20260807 18:19:29.978255 11040 leader_election.cc:288] T 00000000000000000000000000000000 P 981d367211b441f3b7d734291a9a8fde [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:19:29.978529 11046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 981d367211b441f3b7d734291a9a8fde [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:19:29.979391 11046 raft_consensus.cc:686] T 00000000000000000000000000000000 P 981d367211b441f3b7d734291a9a8fde [term 1 LEADER]: Becoming Leader. State: Replica: 981d367211b441f3b7d734291a9a8fde, State: Running, Role: LEADER
I20260807 18:19:29.980077 11046 consensus_queue.cc:237] T 00000000000000000000000000000000 P 981d367211b441f3b7d734291a9a8fde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "981d367211b441f3b7d734291a9a8fde" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 42519 } }
I20260807 18:19:29.980736 11040 sys_catalog.cc:564] T 00000000000000000000000000000000 P 981d367211b441f3b7d734291a9a8fde [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:19:29.983965 11048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 981d367211b441f3b7d734291a9a8fde [sys.catalog]: SysCatalogTable state changed. Reason: New leader 981d367211b441f3b7d734291a9a8fde. Latest consensus state: current_term: 1 leader_uuid: "981d367211b441f3b7d734291a9a8fde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "981d367211b441f3b7d734291a9a8fde" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 42519 } } }
I20260807 18:19:29.983968 11047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 981d367211b441f3b7d734291a9a8fde [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "981d367211b441f3b7d734291a9a8fde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "981d367211b441f3b7d734291a9a8fde" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 42519 } } }
I20260807 18:19:29.984587 11048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 981d367211b441f3b7d734291a9a8fde [sys.catalog]: This master's current role is: LEADER
I20260807 18:19:29.984593 11047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 981d367211b441f3b7d734291a9a8fde [sys.catalog]: This master's current role is: LEADER
I20260807 18:19:29.986541 11055 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:19:29.992660 11055 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:19:30.003300 11055 catalog_manager.cc:1347] Generated new cluster ID: f78168b2b6d54c1aa488c632d06ac930
I20260807 18:19:30.003443 11055 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:19:30.018450 11055 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:19:30.020581 11055 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:19:30.031952 11055 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 981d367211b441f3b7d734291a9a8fde: Generated new TSK 0
I20260807 18:19:30.032984 11055 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:19:30.050027 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.1:0
--local_ip_for_outbound_sockets=127.17.110.1
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:42519
--builtin_ntp_servers=127.17.110.20:45817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:19:30.447464 11068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:19:30.448017 11068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:19:30.448191 11068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:19:30.456100 11068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:19:30.456234 11068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:19:30.456346 11068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.1
I20260807 18:19:30.466081 11068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:45817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:42519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.1
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:19:30.467561 11068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:19:30.469444 11068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:19:30.484927 11074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:30.485039 11076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:30.485061 11079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:19:30.486649 11068 server_base.cc:1034] running on GCE node
I20260807 18:19:30.487535 11068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:19:30.488662 11068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:19:30.489912 11068 hybrid_clock.cc:648] HybridClock initialized: now 1786126770489841 us; error 61 us; skew 500 ppm
I20260807 18:19:30.490401 11068 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:19:30.493145 11068 webserver.cc:460] Webserver started at http://127.17.110.1:46853/ using document root <none> and password file <none>
I20260807 18:19:30.494165 11068 fs_manager.cc:359] Metadata directory not provided
I20260807 18:19:30.494313 11068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:19:30.494689 11068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:19:30.497912 11068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-0/data/instance:
uuid: "bccd627ea4c349d19e0937a0d23f52eb"
format_stamp: "Formatted at 2026-08-07 18:19:30 on dist-test-slave-39l8"
server_key: "ef148fc7805de3f02f453d92160c1b98"
server_key_iv: "a2b3a13bc787f8ba126c9d28632ebde5"
server_key_version: "encryptionkey@0"
I20260807 18:19:30.498889 11068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance:
uuid: "bccd627ea4c349d19e0937a0d23f52eb"
format_stamp: "Formatted at 2026-08-07 18:19:30 on dist-test-slave-39l8"
server_key: "ef148fc7805de3f02f453d92160c1b98"
server_key_iv: "a2b3a13bc787f8ba126c9d28632ebde5"
server_key_version: "encryptionkey@0"
I20260807 18:19:30.505117 11068 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260807 18:19:30.509459 11085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:30.511405 11068 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:19:30.511600 11068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-0/wal
uuid: "bccd627ea4c349d19e0937a0d23f52eb"
format_stamp: "Formatted at 2026-08-07 18:19:30 on dist-test-slave-39l8"
server_key: "ef148fc7805de3f02f453d92160c1b98"
server_key_iv: "a2b3a13bc787f8ba126c9d28632ebde5"
server_key_version: "encryptionkey@0"
I20260807 18:19:30.511857 11068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:19:30.532063 11068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:19:30.533174 11068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:19:30.533533 11068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:19:30.534901 11068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:19:30.536770 11068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:19:30.536867 11068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:30.536947 11068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:19:30.537006 11068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:30.573887 11068 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.1:36441
I20260807 18:19:30.573932 11250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.1:36441 every 8 connection(s)
I20260807 18:19:30.575556 11068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
I20260807 18:19:30.584029 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 11068
I20260807 18:19:30.584264 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance
I20260807 18:19:30.584740 17848 external_mini_cluster.cc:1428] Setting key c53ea5edaa77c9da056f17b83c2631b2
I20260807 18:19:30.587904 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.2:0
--local_ip_for_outbound_sockets=127.17.110.2
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:42519
--builtin_ntp_servers=127.17.110.20:45817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:19:30.595559 11257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:30.578612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:40149 (local address 127.17.110.62:42519)
0807 18:19:30.579247 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:30.579252 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:30.580539 (+ 1287us) server_negotiation.cc:407] Connection header received
0807 18:19:30.582631 (+ 2092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:30.582637 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:30.582800 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:30.582926 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:30.584188 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:30.585090 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:30.587234 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:30.587670 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:30.587814 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:30.594307 (+ 6493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:30.594314 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:30.594317 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:19:30.594342 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:30.594387 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:19:30.594401 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:30.594527 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:30.594918 (+ 391us) server_negotiation.cc:299] Negotiation successful
0807 18:19:30.595066 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":384,"threads_started":1}
I20260807 18:19:30.596652 11259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:30.579242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42519 (local address 127.17.110.1:40149)
0807 18:19:30.580331 (+ 1089us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:30.580372 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:30.582282 (+ 1910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:30.583218 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:30.583227 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:30.583331 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:19:30.583985 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:30.584005 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:30.585290 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:30.585298 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:30.587037 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:30.587051 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:30.587932 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:30.587940 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:30.588273 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:30.593797 (+ 5524us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:19:30.593801 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:19:30.593858 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:19:30.593973 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:19:30.594525 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:19:30.594545 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:19:30.594558 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 18:19:30.594821 (+ 263us) client_negotiation.cc:240] Negotiation successful
0807 18:19:30.595278 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":333,"threads_started":1}
I20260807 18:19:30.598663 11252 heartbeater.cc:347] Connected to a master server at 127.17.110.62:42519
I20260807 18:19:30.599077 11252 heartbeater.cc:464] Registering TS with master...
I20260807 18:19:30.599931 11252 heartbeater.cc:511] Master 127.17.110.62:42519 requested a full tablet report, sending...
I20260807 18:19:30.602516 10983 ts_manager.cc:194] Registered new tserver with Master: bccd627ea4c349d19e0937a0d23f52eb (127.17.110.1:36441)
I20260807 18:19:30.604267 10983 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.110.1:40149
W20260807 18:19:30.975557 11260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:19:30.975884 11260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:19:30.975976 11260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:19:30.983615 11260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:19:30.983731 11260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:19:30.983842 11260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.2
I20260807 18:19:30.993769 11260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:45817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:42519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.2
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:19:30.995268 11260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:19:30.997212 11260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:19:31.012506 11272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:31.012594 11270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:31.012604 11269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:19:31.013499 11260 server_base.cc:1034] running on GCE node
I20260807 18:19:31.014247 11260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:19:31.015415 11260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:19:31.016675 11260 hybrid_clock.cc:648] HybridClock initialized: now 1786126771016580 us; error 49 us; skew 500 ppm
I20260807 18:19:31.017050 11260 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:19:31.019814 11260 webserver.cc:460] Webserver started at http://127.17.110.2:42269/ using document root <none> and password file <none>
I20260807 18:19:31.020609 11260 fs_manager.cc:359] Metadata directory not provided
I20260807 18:19:31.020744 11260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:19:31.021106 11260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:19:31.024433 11260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-1/data/instance:
uuid: "33d215770177400ba9ab2dfe06647979"
format_stamp: "Formatted at 2026-08-07 18:19:31 on dist-test-slave-39l8"
server_key: "41930d5ff77acd9a6e80ccc651366922"
server_key_iv: "90b271ea6b77a350ff26cd10ff87a1b6"
server_key_version: "encryptionkey@0"
I20260807 18:19:31.025337 11260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance:
uuid: "33d215770177400ba9ab2dfe06647979"
format_stamp: "Formatted at 2026-08-07 18:19:31 on dist-test-slave-39l8"
server_key: "41930d5ff77acd9a6e80ccc651366922"
server_key_iv: "90b271ea6b77a350ff26cd10ff87a1b6"
server_key_version: "encryptionkey@0"
I20260807 18:19:31.031318 11260 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 18:19:31.035672 11281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:31.037209 11260 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:19:31.037436 11260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-1/wal
uuid: "33d215770177400ba9ab2dfe06647979"
format_stamp: "Formatted at 2026-08-07 18:19:31 on dist-test-slave-39l8"
server_key: "41930d5ff77acd9a6e80ccc651366922"
server_key_iv: "90b271ea6b77a350ff26cd10ff87a1b6"
server_key_version: "encryptionkey@0"
I20260807 18:19:31.037712 11260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:19:31.052155 11260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:19:31.053089 11260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:19:31.053421 11260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:19:31.054491 11260 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:19:31.056123 11260 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:19:31.056262 11260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:31.056389 11260 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:19:31.056478 11260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:31.093787 11260 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.2:36199
I20260807 18:19:31.093927 11454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.2:36199 every 8 connection(s)
I20260807 18:19:31.095381 11260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
I20260807 18:19:31.104512 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 11260
I20260807 18:19:31.104713 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance
I20260807 18:19:31.105260 17848 external_mini_cluster.cc:1428] Setting key 6bb92775dd50e7b044aae6ec7b1c4308
I20260807 18:19:31.108454 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.3:0
--local_ip_for_outbound_sockets=127.17.110.3
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:42519
--builtin_ntp_servers=127.17.110.20:45817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:19:31.110049 11457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:31.098440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:54443 (local address 127.17.110.62:42519)
0807 18:19:31.099121 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:31.099126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:31.100336 (+ 1210us) server_negotiation.cc:407] Connection header received
0807 18:19:31.102229 (+ 1893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:31.102234 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:31.102332 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:31.102495 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:31.103766 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:31.104847 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:31.106852 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:31.107389 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:31.107541 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:31.109121 (+ 1580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:31.109126 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:31.109129 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:19:31.109148 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:31.109192 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:19:31.109204 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:31.109305 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:31.109556 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 18:19:31.109661 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":357,"threads_started":1}
I20260807 18:19:31.111204 11459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:31.099147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42519 (local address 127.17.110.2:54443)
0807 18:19:31.100128 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:31.100196 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:31.101926 (+ 1730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:31.102823 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:31.102833 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:31.102930 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:19:31.103564 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:31.103584 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:31.105020 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:31.105025 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:31.106618 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:31.106633 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:31.107649 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:31.107657 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:31.107923 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:31.108753 (+ 830us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:19:31.108756 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:19:31.108802 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:19:31.108894 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:19:31.109307 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:19:31.109321 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:19:31.109332 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 18:19:31.109529 (+ 197us) client_negotiation.cc:240] Negotiation successful
0807 18:19:31.109981 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":291,"threads_started":1}
I20260807 18:19:31.112897 11455 heartbeater.cc:347] Connected to a master server at 127.17.110.62:42519
I20260807 18:19:31.113250 11455 heartbeater.cc:464] Registering TS with master...
I20260807 18:19:31.114255 11455 heartbeater.cc:511] Master 127.17.110.62:42519 requested a full tablet report, sending...
I20260807 18:19:31.116217 10983 ts_manager.cc:194] Registered new tserver with Master: 33d215770177400ba9ab2dfe06647979 (127.17.110.2:36199)
I20260807 18:19:31.117110 10983 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.110.2:54443
W20260807 18:19:31.489703 11460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:19:31.490077 11460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:19:31.490243 11460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:19:31.498001 11460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:19:31.498184 11460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:19:31.498373 11460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.3
I20260807 18:19:31.508383 11460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:45817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:42519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.3
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:19:31.509793 11460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:19:31.511816 11460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:19:31.526554 11466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:31.526597 11472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:31.526525 11467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:19:31.527695 11460 server_base.cc:1034] running on GCE node
I20260807 18:19:31.528484 11460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:19:31.529656 11460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:19:31.530926 11460 hybrid_clock.cc:648] HybridClock initialized: now 1786126771530879 us; error 75 us; skew 500 ppm
I20260807 18:19:31.531292 11460 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:19:31.538443 11460 webserver.cc:460] Webserver started at http://127.17.110.3:40771/ using document root <none> and password file <none>
I20260807 18:19:31.539381 11460 fs_manager.cc:359] Metadata directory not provided
I20260807 18:19:31.539544 11460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:19:31.539911 11460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:19:31.543562 11460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-2/data/instance:
uuid: "7fdb0a8077034b7cb1d3542e7b6f3acb"
format_stamp: "Formatted at 2026-08-07 18:19:31 on dist-test-slave-39l8"
server_key: "fff1c267dcaa5844f7b81a7f289e3b18"
server_key_iv: "5e250fd4d866193084bd78f21a690fc3"
server_key_version: "encryptionkey@0"
I20260807 18:19:31.544536 11460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance:
uuid: "7fdb0a8077034b7cb1d3542e7b6f3acb"
format_stamp: "Formatted at 2026-08-07 18:19:31 on dist-test-slave-39l8"
server_key: "fff1c267dcaa5844f7b81a7f289e3b18"
server_key_iv: "5e250fd4d866193084bd78f21a690fc3"
server_key_version: "encryptionkey@0"
I20260807 18:19:31.550870 11460 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260807 18:19:31.555097 11484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:31.556746 11460 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:19:31.556990 11460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-2/wal
uuid: "7fdb0a8077034b7cb1d3542e7b6f3acb"
format_stamp: "Formatted at 2026-08-07 18:19:31 on dist-test-slave-39l8"
server_key: "fff1c267dcaa5844f7b81a7f289e3b18"
server_key_iv: "5e250fd4d866193084bd78f21a690fc3"
server_key_version: "encryptionkey@0"
I20260807 18:19:31.557255 11460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:19:31.576458 11460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:19:31.577395 11460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:19:31.577744 11460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:19:31.579032 11460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:19:31.580636 11460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:19:31.580797 11460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:31.580942 11460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:19:31.581043 11460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:31.608215 11252 heartbeater.cc:503] Master 127.17.110.62:42519 was elected leader, sending a full tablet report...
I20260807 18:19:31.619009 11460 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.3:44547
I20260807 18:19:31.619122 11658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.3:44547 every 8 connection(s)
I20260807 18:19:31.620463 11460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
I20260807 18:19:31.625275 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 11460
I20260807 18:19:31.625499 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance
I20260807 18:19:31.626281 17848 external_mini_cluster.cc:1428] Setting key d5dbe84df680726edd92305502b41132
I20260807 18:19:31.636897 11661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:31.623467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:39181 (local address 127.17.110.62:42519)
0807 18:19:31.623987 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:31.623992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:31.625800 (+ 1808us) server_negotiation.cc:407] Connection header received
0807 18:19:31.628396 (+ 2596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:31.628403 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:31.628502 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:31.628605 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:31.630275 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:31.631161 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:31.633081 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:31.633843 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:31.633986 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:31.635860 (+ 1874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:31.635866 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:31.635869 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:19:31.635893 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:31.635975 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:19:31.635997 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:31.636144 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:31.636476 (+ 332us) server_negotiation.cc:299] Negotiation successful
0807 18:19:31.636586 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":277,"threads_started":1}
I20260807 18:19:31.638113 11662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:31.624168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:42519 (local address 127.17.110.3:39181)
0807 18:19:31.625522 (+ 1354us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:31.625600 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:31.627847 (+ 2247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:31.628968 (+ 1121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:31.628978 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:31.629085 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:19:31.629987 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:31.630036 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:31.631393 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:31.631398 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:31.632794 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:31.632812 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:31.634193 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:31.634201 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:31.634470 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:31.635442 (+ 972us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:19:31.635445 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:19:31.635495 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:19:31.635596 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:19:31.636160 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:19:31.636173 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:19:31.636181 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 18:19:31.636401 (+ 220us) client_negotiation.cc:240] Negotiation successful
0807 18:19:31.636815 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":408,"threads_started":1}
I20260807 18:19:31.639891 11659 heartbeater.cc:347] Connected to a master server at 127.17.110.62:42519
I20260807 18:19:31.640261 11659 heartbeater.cc:464] Registering TS with master...
I20260807 18:19:31.641429 11659 heartbeater.cc:511] Master 127.17.110.62:42519 requested a full tablet report, sending...
I20260807 18:19:31.643342 10983 ts_manager.cc:194] Registered new tserver with Master: 7fdb0a8077034b7cb1d3542e7b6f3acb (127.17.110.3:44547)
I20260807 18:19:31.644181 10983 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.110.3:39181
I20260807 18:19:31.656363 17848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:19:31.668081 11661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:31.661206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37768 (local address 127.17.110.62:42519)
0807 18:19:31.661472 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:31.661483 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:31.661964 (+ 481us) server_negotiation.cc:407] Connection header received
0807 18:19:31.662179 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:31.662184 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:31.662254 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:31.662394 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:31.663045 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:31.663836 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:31.665446 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:31.665892 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:31.666009 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:31.667232 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:31.667238 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:31.667241 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:19:31.667263 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:31.667307 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:19:31.667320 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:31.667408 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:31.667606 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 18:19:31.667696 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260807 18:19:31.672995 10983 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37768:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:19:31.680730 10983 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:19:31.713128 11683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:31.695563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49506 (local address 127.17.110.1:36441)
0807 18:19:31.696569 (+ 1006us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:31.696583 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:31.700610 (+ 4027us) server_negotiation.cc:407] Connection header received
0807 18:19:31.701123 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:31.701136 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:31.701489 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:31.701721 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:19:31.703197 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:31.705864 (+ 2667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:31.709744 (+ 3880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:31.711491 (+ 1747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:31.711648 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:31.711992 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:31.712474 (+ 482us) server_negotiation.cc:299] Negotiation successful
0807 18:19:31.712699 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":768,"thread_start_us":589,"threads_started":1}
I20260807 18:19:31.713184 11682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:31.695339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:36441 (local address 127.0.0.1:49506)
0807 18:19:31.700400 (+ 5061us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:31.700465 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:31.700797 (+ 332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:31.702223 (+ 1426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:31.702234 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:31.702281 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:19:31.702894 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:31.702915 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:31.706235 (+ 3320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:31.706244 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:31.709422 (+ 3178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:31.709436 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:31.711747 (+ 2311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:31.711757 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:31.712067 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:31.712207 (+ 140us) client_negotiation.cc:769] Sending connection context
0807 18:19:31.712377 (+ 170us) client_negotiation.cc:240] Negotiation successful
0807 18:19:31.712728 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4707,"mutex_wait_us":104,"thread_start_us":11909,"threads_started":1}
I20260807 18:19:31.717799 11686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:31.699996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48418 (local address 127.17.110.2:36199)
0807 18:19:31.707204 (+ 7208us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:31.707222 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:31.708440 (+ 1218us) server_negotiation.cc:407] Connection header received
0807 18:19:31.708585 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:31.708593 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:31.708838 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:31.709056 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:19:31.710179 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:31.711705 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:31.714966 (+ 3261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:31.716519 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:31.716671 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:31.716909 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:31.717273 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 18:19:31.717448 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6952,"thread_start_us":6774,"threads_started":1}
I20260807 18:19:31.717824 11687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:31.699997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:36199 (local address 127.0.0.1:48418)
0807 18:19:31.708110 (+ 8113us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:31.708134 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:31.708289 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:31.709331 (+ 1042us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:31.709342 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:31.709372 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:19:31.709936 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:31.709976 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:31.711891 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:31.711896 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:31.714613 (+ 2717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:31.714630 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:31.716795 (+ 2165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:31.716805 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:31.717023 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:31.717080 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 18:19:31.717198 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 18:19:31.717365 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7908,"mutex_wait_us":518,"thread_start_us":7177,"threads_started":1}
I20260807 18:19:31.717943 11681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:31.695415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:44547 (local address 127.0.0.1:41976)
0807 18:19:31.707666 (+ 12251us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:31.707701 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:31.708039 (+ 338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:31.709241 (+ 1202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:31.709248 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:31.709276 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:19:31.709857 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:31.709872 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:31.711459 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:31.711468 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:31.714662 (+ 3194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:31.714679 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:31.717087 (+ 2408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:31.717093 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:31.717317 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:31.717352 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 18:19:31.717438 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 18:19:31.717575 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12006,"mutex_wait_us":75,"thread_start_us":4830,"threads_started":1}
I20260807 18:19:31.718798 11685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:31.696007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41976 (local address 127.17.110.3:44547)
0807 18:19:31.699674 (+ 3667us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:31.699684 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:31.707861 (+ 8177us) server_negotiation.cc:407] Connection header received
0807 18:19:31.708376 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:31.708387 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:31.708679 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:31.708889 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:19:31.710088 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:31.711308 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:31.715028 (+ 3720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:31.716439 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:31.716632 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:31.716950 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:31.718239 (+ 1289us) server_negotiation.cc:299] Negotiation successful
0807 18:19:31.718409 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3478,"thread_start_us":3336,"threads_started":1}
I20260807 18:19:31.718215 11158 tablet_service.cc:1467] Processing CreateTablet for tablet 0eeddbaed81b4419ba946a691b5e158c (DEFAULT_TABLE table=test-table [id=b18dfa5e4ca34551bcd27e494e63abbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:19:31.720135 11158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0eeddbaed81b4419ba946a691b5e158c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:19:31.723253 11562 tablet_service.cc:1467] Processing CreateTablet for tablet 0eeddbaed81b4419ba946a691b5e158c (DEFAULT_TABLE table=test-table [id=b18dfa5e4ca34551bcd27e494e63abbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:19:31.723258 11351 tablet_service.cc:1467] Processing CreateTablet for tablet 0eeddbaed81b4419ba946a691b5e158c (DEFAULT_TABLE table=test-table [id=b18dfa5e4ca34551bcd27e494e63abbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:19:31.724980 11562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0eeddbaed81b4419ba946a691b5e158c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:19:31.724978 11351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0eeddbaed81b4419ba946a691b5e158c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:19:31.732924 11688 tablet_bootstrap.cc:492] T 0eeddbaed81b4419ba946a691b5e158c P bccd627ea4c349d19e0937a0d23f52eb: Bootstrap starting.
I20260807 18:19:31.736652 11688 tablet_bootstrap.cc:654] T 0eeddbaed81b4419ba946a691b5e158c P bccd627ea4c349d19e0937a0d23f52eb: Neither blocks nor log segments found. Creating new log.
I20260807 18:19:31.739049 11688 log.cc:824] T 0eeddbaed81b4419ba946a691b5e158c P bccd627ea4c349d19e0937a0d23f52eb: Log is configured to *not* fsync() on all Append() calls
I20260807 18:19:31.742199 11688 tablet_bootstrap.cc:492] T 0eeddbaed81b4419ba946a691b5e158c P bccd627ea4c349d19e0937a0d23f52eb: No bootstrap required, opened a new log
I20260807 18:19:31.742534 11688 ts_tablet_manager.cc:1397] T 0eeddbaed81b4419ba946a691b5e158c P bccd627ea4c349d19e0937a0d23f52eb: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260807 18:19:31.743099 11694 tablet_bootstrap.cc:492] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979: Bootstrap starting.
I20260807 18:19:31.745249 11696 tablet_bootstrap.cc:492] T 0eeddbaed81b4419ba946a691b5e158c P 7fdb0a8077034b7cb1d3542e7b6f3acb: Bootstrap starting.
I20260807 18:19:31.747392 11694 tablet_bootstrap.cc:654] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979: Neither blocks nor log segments found. Creating new log.
I20260807 18:19:31.748850 11694 log.cc:824] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979: Log is configured to *not* fsync() on all Append() calls
I20260807 18:19:31.749114 11696 tablet_bootstrap.cc:654] T 0eeddbaed81b4419ba946a691b5e158c P 7fdb0a8077034b7cb1d3542e7b6f3acb: Neither blocks nor log segments found. Creating new log.
I20260807 18:19:31.750368 11696 log.cc:824] T 0eeddbaed81b4419ba946a691b5e158c P 7fdb0a8077034b7cb1d3542e7b6f3acb: Log is configured to *not* fsync() on all Append() calls
I20260807 18:19:31.753804 11694 tablet_bootstrap.cc:492] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979: No bootstrap required, opened a new log
I20260807 18:19:31.753976 11696 tablet_bootstrap.cc:492] T 0eeddbaed81b4419ba946a691b5e158c P 7fdb0a8077034b7cb1d3542e7b6f3acb: No bootstrap required, opened a new log
I20260807 18:19:31.754181 11694 ts_tablet_manager.cc:1397] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260807 18:19:31.754266 11696 ts_tablet_manager.cc:1397] T 0eeddbaed81b4419ba946a691b5e158c P 7fdb0a8077034b7cb1d3542e7b6f3acb: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260807 18:19:31.754578 11688 raft_consensus.cc:348] T 0eeddbaed81b4419ba946a691b5e158c P bccd627ea4c349d19e0937a0d23f52eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bccd627ea4c349d19e0937a0d23f52eb" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 36441 } } peers { permanent_uuid: "7fdb0a8077034b7cb1d3542e7b6f3acb" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44547 } } peers { permanent_uuid: "33d215770177400ba9ab2dfe06647979" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 36199 } }
I20260807 18:19:31.755120 11688 raft_consensus.cc:374] T 0eeddbaed81b4419ba946a691b5e158c P bccd627ea4c349d19e0937a0d23f52eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:19:31.755244 11688 raft_consensus.cc:729] T 0eeddbaed81b4419ba946a691b5e158c P bccd627ea4c349d19e0937a0d23f52eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bccd627ea4c349d19e0937a0d23f52eb, State: Initialized, Role: FOLLOWER
I20260807 18:19:31.755924 11688 consensus_queue.cc:260] T 0eeddbaed81b4419ba946a691b5e158c P bccd627ea4c349d19e0937a0d23f52eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bccd627ea4c349d19e0937a0d23f52eb" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 36441 } } peers { permanent_uuid: "7fdb0a8077034b7cb1d3542e7b6f3acb" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44547 } } peers { permanent_uuid: "33d215770177400ba9ab2dfe06647979" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 36199 } }
I20260807 18:19:31.764427 11688 ts_tablet_manager.cc:1428] T 0eeddbaed81b4419ba946a691b5e158c P bccd627ea4c349d19e0937a0d23f52eb: Time spent starting tablet: real 0.022s user 0.020s sys 0.000s
I20260807 18:19:31.764219 11694 raft_consensus.cc:348] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bccd627ea4c349d19e0937a0d23f52eb" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 36441 } } peers { permanent_uuid: "7fdb0a8077034b7cb1d3542e7b6f3acb" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44547 } } peers { permanent_uuid: "33d215770177400ba9ab2dfe06647979" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 36199 } }
I20260807 18:19:31.764756 11694 raft_consensus.cc:374] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:19:31.764880 11694 raft_consensus.cc:729] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d215770177400ba9ab2dfe06647979, State: Initialized, Role: FOLLOWER
I20260807 18:19:31.765647 11694 consensus_queue.cc:260] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bccd627ea4c349d19e0937a0d23f52eb" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 36441 } } peers { permanent_uuid: "7fdb0a8077034b7cb1d3542e7b6f3acb" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44547 } } peers { permanent_uuid: "33d215770177400ba9ab2dfe06647979" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 36199 } }
I20260807 18:19:31.766944 11696 raft_consensus.cc:348] T 0eeddbaed81b4419ba946a691b5e158c P 7fdb0a8077034b7cb1d3542e7b6f3acb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bccd627ea4c349d19e0937a0d23f52eb" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 36441 } } peers { permanent_uuid: "7fdb0a8077034b7cb1d3542e7b6f3acb" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44547 } } peers { permanent_uuid: "33d215770177400ba9ab2dfe06647979" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 36199 } }
I20260807 18:19:31.767313 11455 heartbeater.cc:503] Master 127.17.110.62:42519 was elected leader, sending a full tablet report...
I20260807 18:19:31.767448 11696 raft_consensus.cc:374] T 0eeddbaed81b4419ba946a691b5e158c P 7fdb0a8077034b7cb1d3542e7b6f3acb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:19:31.767555 11696 raft_consensus.cc:729] T 0eeddbaed81b4419ba946a691b5e158c P 7fdb0a8077034b7cb1d3542e7b6f3acb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fdb0a8077034b7cb1d3542e7b6f3acb, State: Initialized, Role: FOLLOWER
I20260807 18:19:31.768201 11694 ts_tablet_manager.cc:1428] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979: Time spent starting tablet: real 0.014s user 0.004s sys 0.010s
I20260807 18:19:31.768401 11696 consensus_queue.cc:260] T 0eeddbaed81b4419ba946a691b5e158c P 7fdb0a8077034b7cb1d3542e7b6f3acb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bccd627ea4c349d19e0937a0d23f52eb" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 36441 } } peers { permanent_uuid: "7fdb0a8077034b7cb1d3542e7b6f3acb" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44547 } } peers { permanent_uuid: "33d215770177400ba9ab2dfe06647979" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 36199 } }
I20260807 18:19:31.770009 11659 heartbeater.cc:503] Master 127.17.110.62:42519 was elected leader, sending a full tablet report...
I20260807 18:19:31.770787 11696 ts_tablet_manager.cc:1428] T 0eeddbaed81b4419ba946a691b5e158c P 7fdb0a8077034b7cb1d3542e7b6f3acb: Time spent starting tablet: real 0.016s user 0.017s sys 0.002s
W20260807 18:19:31.828451 11256 tablet.cc:2369] T 0eeddbaed81b4419ba946a691b5e158c P bccd627ea4c349d19e0937a0d23f52eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:19:31.847867 11456 tablet.cc:2369] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:19:31.872380 11660 tablet.cc:2369] T 0eeddbaed81b4419ba946a691b5e158c P 7fdb0a8077034b7cb1d3542e7b6f3acb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:19:31.950759 11703 raft_consensus.cc:482] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:19:31.951241 11703 raft_consensus.cc:504] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bccd627ea4c349d19e0937a0d23f52eb" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 36441 } } peers { permanent_uuid: "7fdb0a8077034b7cb1d3542e7b6f3acb" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44547 } } peers { permanent_uuid: "33d215770177400ba9ab2dfe06647979" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 36199 } }
I20260807 18:19:31.953272 11703 leader_election.cc:288] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bccd627ea4c349d19e0937a0d23f52eb (127.17.110.1:36441), 7fdb0a8077034b7cb1d3542e7b6f3acb (127.17.110.3:44547)
I20260807 18:19:31.960868 11685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:31.953819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:38993 (local address 127.17.110.3:44547)
0807 18:19:31.954072 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:31.954082 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:31.954894 (+ 812us) server_negotiation.cc:407] Connection header received
0807 18:19:31.955114 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:31.955119 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:31.955182 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:31.955288 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:19:31.955936 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:31.956697 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:31.958780 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:31.959867 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:31.960076 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:31.960152 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:31.960454 (+ 302us) server_negotiation.cc:299] Negotiation successful
0807 18:19:31.960592 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 18:19:31.960971 11683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:31.953697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:50209 (local address 127.17.110.1:36441)
0807 18:19:31.953947 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:31.953955 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:31.954814 (+ 859us) server_negotiation.cc:407] Connection header received
0807 18:19:31.955065 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:31.955126 (+ 61us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:31.955213 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:31.955344 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:19:31.956098 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:31.956848 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:31.959012 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:31.959968 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:31.960194 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:31.960269 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:31.960557 (+ 288us) server_negotiation.cc:299] Negotiation successful
0807 18:19:31.960701 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260807 18:19:31.961244 11706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:31.953852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:44547 (local address 127.17.110.2:38993)
0807 18:19:31.954765 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:31.954795 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:31.954951 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:31.955303 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:31.955308 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:31.955332 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:19:31.955746 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:31.955756 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:31.956838 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:31.956847 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:31.958555 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:31.958570 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:31.960089 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:31.960095 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:31.960277 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:31.960303 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:19:31.960413 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 18:19:31.960554 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"mutex_wait_us":26,"thread_start_us":527,"threads_started":1}
I20260807 18:19:31.961272 11705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:31.953660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:36441 (local address 127.17.110.2:50209)
0807 18:19:31.954627 (+ 967us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:31.954657 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:31.954896 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:31.955356 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:31.955361 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:31.955392 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:19:31.955917 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:31.955929 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:31.957025 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:31.957038 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:31.958814 (+ 1776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:31.958829 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:31.960201 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:31.960208 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:31.960388 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:31.960414 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:19:31.960527 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 18:19:31.960697 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":789,"mutex_wait_us":4,"thread_start_us":525,"threads_started":1}
I20260807 18:19:31.962435 11187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eeddbaed81b4419ba946a691b5e158c" candidate_uuid: "33d215770177400ba9ab2dfe06647979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bccd627ea4c349d19e0937a0d23f52eb" is_pre_election: true
I20260807 18:19:31.962426 11595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eeddbaed81b4419ba946a691b5e158c" candidate_uuid: "33d215770177400ba9ab2dfe06647979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fdb0a8077034b7cb1d3542e7b6f3acb" is_pre_election: true
I20260807 18:19:31.963059 11187 raft_consensus.cc:2454] T 0eeddbaed81b4419ba946a691b5e158c P bccd627ea4c349d19e0937a0d23f52eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d215770177400ba9ab2dfe06647979 in term 0.
I20260807 18:19:31.963080 11595 raft_consensus.cc:2454] T 0eeddbaed81b4419ba946a691b5e158c P 7fdb0a8077034b7cb1d3542e7b6f3acb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d215770177400ba9ab2dfe06647979 in term 0.
I20260807 18:19:31.963773 11287 leader_election.cc:302] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d215770177400ba9ab2dfe06647979, bccd627ea4c349d19e0937a0d23f52eb; no voters:
I20260807 18:19:31.964203 11703 raft_consensus.cc:2781] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:19:31.964363 11703 raft_consensus.cc:482] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:19:31.964471 11703 raft_consensus.cc:3037] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:19:31.966840 11703 raft_consensus.cc:504] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bccd627ea4c349d19e0937a0d23f52eb" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 36441 } } peers { permanent_uuid: "7fdb0a8077034b7cb1d3542e7b6f3acb" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44547 } } peers { permanent_uuid: "33d215770177400ba9ab2dfe06647979" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 36199 } }
I20260807 18:19:31.967561 11703 leader_election.cc:288] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979 [CANDIDATE]: Term 1 election: Requested vote from peers bccd627ea4c349d19e0937a0d23f52eb (127.17.110.1:36441), 7fdb0a8077034b7cb1d3542e7b6f3acb (127.17.110.3:44547)
I20260807 18:19:31.968128 11187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eeddbaed81b4419ba946a691b5e158c" candidate_uuid: "33d215770177400ba9ab2dfe06647979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bccd627ea4c349d19e0937a0d23f52eb"
I20260807 18:19:31.968187 11595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eeddbaed81b4419ba946a691b5e158c" candidate_uuid: "33d215770177400ba9ab2dfe06647979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fdb0a8077034b7cb1d3542e7b6f3acb"
I20260807 18:19:31.968380 11187 raft_consensus.cc:3037] T 0eeddbaed81b4419ba946a691b5e158c P bccd627ea4c349d19e0937a0d23f52eb [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:19:31.968441 11595 raft_consensus.cc:3037] T 0eeddbaed81b4419ba946a691b5e158c P 7fdb0a8077034b7cb1d3542e7b6f3acb [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:19:31.970809 11187 raft_consensus.cc:2454] T 0eeddbaed81b4419ba946a691b5e158c P bccd627ea4c349d19e0937a0d23f52eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d215770177400ba9ab2dfe06647979 in term 1.
I20260807 18:19:31.971000 11595 raft_consensus.cc:2454] T 0eeddbaed81b4419ba946a691b5e158c P 7fdb0a8077034b7cb1d3542e7b6f3acb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d215770177400ba9ab2dfe06647979 in term 1.
I20260807 18:19:31.971453 11287 leader_election.cc:302] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d215770177400ba9ab2dfe06647979, bccd627ea4c349d19e0937a0d23f52eb; no voters:
I20260807 18:19:31.971818 11703 raft_consensus.cc:2781] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:19:31.972409 11703 raft_consensus.cc:686] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979 [term 1 LEADER]: Becoming Leader. State: Replica: 33d215770177400ba9ab2dfe06647979, State: Running, Role: LEADER
I20260807 18:19:31.973029 11703 consensus_queue.cc:237] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bccd627ea4c349d19e0937a0d23f52eb" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 36441 } } peers { permanent_uuid: "7fdb0a8077034b7cb1d3542e7b6f3acb" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44547 } } peers { permanent_uuid: "33d215770177400ba9ab2dfe06647979" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 36199 } }
I20260807 18:19:31.978658 10982 catalog_manager.cc:5495] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33d215770177400ba9ab2dfe06647979 (127.17.110.2). New cstate: current_term: 1 leader_uuid: "33d215770177400ba9ab2dfe06647979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bccd627ea4c349d19e0937a0d23f52eb" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 36441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fdb0a8077034b7cb1d3542e7b6f3acb" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d215770177400ba9ab2dfe06647979" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 36199 } health_report { overall_health: HEALTHY } } }
I20260807 18:19:32.010334 11686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:32.003981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48426 (local address 127.17.110.2:36199)
0807 18:19:32.004317 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:32.004323 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:32.004345 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:19:32.004612 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:32.004617 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:32.004714 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:32.004887 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:19:32.005297 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:32.006006 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:32.007597 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:32.008023 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:32.008222 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:32.009005 (+ 783us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:19:32.009579 (+ 574us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:19:32.009717 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:32.009896 (+ 179us) server_negotiation.cc:299] Negotiation successful
0807 18:19:32.009977 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196}
I20260807 18:19:32.028880 11595 raft_consensus.cc:1261] T 0eeddbaed81b4419ba946a691b5e158c P 7fdb0a8077034b7cb1d3542e7b6f3acb [term 1 FOLLOWER]: Refusing update from remote peer 33d215770177400ba9ab2dfe06647979: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:19:32.028910 11187 raft_consensus.cc:1261] T 0eeddbaed81b4419ba946a691b5e158c P bccd627ea4c349d19e0937a0d23f52eb [term 1 FOLLOWER]: Refusing update from remote peer 33d215770177400ba9ab2dfe06647979: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:19:32.030316 11703 consensus_queue.cc:1035] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bccd627ea4c349d19e0937a0d23f52eb" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 36441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:19:32.030731 11707 consensus_queue.cc:1035] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fdb0a8077034b7cb1d3542e7b6f3acb" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:19:32.044902 11715 mvcc.cc:205] Tried to move back new op lower bound from 7315975258215313408 to 7315975258006212608. Current Snapshot: MvccSnapshot[applied={T|T < 7315975258215313408}]
I20260807 18:19:32.047829 11714 mvcc.cc:205] Tried to move back new op lower bound from 7315975258215313408 to 7315975258006212608. Current Snapshot: MvccSnapshot[applied={T|T < 7315975258215313408 or (T in {7315975258215313408})}]
I20260807 18:19:32.057717 10983 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:19:32.058123 10983 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:19:32.066605 10983 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 981d367211b441f3b7d734291a9a8fde: Sending DeleteTablet for 3 replicas of tablet 0eeddbaed81b4419ba946a691b5e158c
I20260807 18:19:32.068876 11351 tablet_service.cc:1514] Processing DeleteTablet for tablet 0eeddbaed81b4419ba946a691b5e158c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:19:32 UTC) from {username='slave'} at 127.0.0.1:48418
I20260807 18:19:32.072386 11158 tablet_service.cc:1514] Processing DeleteTablet for tablet 0eeddbaed81b4419ba946a691b5e158c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:19:32 UTC) from {username='slave'} at 127.0.0.1:49506
I20260807 18:19:32.075488 11728 tablet_replica.cc:330] T 0eeddbaed81b4419ba946a691b5e158c P bccd627ea4c349d19e0937a0d23f52eb: stopping tablet replica
I20260807 18:19:32.075981 11728 raft_consensus.cc:2229] T 0eeddbaed81b4419ba946a691b5e158c P bccd627ea4c349d19e0937a0d23f52eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:19:32.076531 11728 raft_consensus.cc:2258] T 0eeddbaed81b4419ba946a691b5e158c P bccd627ea4c349d19e0937a0d23f52eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:19:32.076699 11562 tablet_service.cc:1514] Processing DeleteTablet for tablet 0eeddbaed81b4419ba946a691b5e158c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:19:32 UTC) from {username='slave'} at 127.0.0.1:41976
I20260807 18:19:32.078846 11350 tablet_service.cc:1514] Processing DeleteTablet for tablet 0eeddbaed81b4419ba946a691b5e158c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:19:32 UTC) from {username='slave'} at 127.0.0.1:48418
I20260807 18:19:32.079046 11728 ts_tablet_manager.cc:1905] T 0eeddbaed81b4419ba946a691b5e158c P bccd627ea4c349d19e0937a0d23f52eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:19:32.081205 11727 tablet_replica.cc:330] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979: stopping tablet replica
I20260807 18:19:32.081877 11729 tablet_replica.cc:330] T 0eeddbaed81b4419ba946a691b5e158c P 7fdb0a8077034b7cb1d3542e7b6f3acb: stopping tablet replica
I20260807 18:19:32.081898 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 11068
I20260807 18:19:32.082341 11727 raft_consensus.cc:2229] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:19:32.082363 11729 raft_consensus.cc:2229] T 0eeddbaed81b4419ba946a691b5e158c P 7fdb0a8077034b7cb1d3542e7b6f3acb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:19:32.082931 11729 raft_consensus.cc:2258] T 0eeddbaed81b4419ba946a691b5e158c P 7fdb0a8077034b7cb1d3542e7b6f3acb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:19:32.083249 11727 raft_consensus.cc:2258] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:19:32.089746 11728 ts_tablet_manager.cc:1918] T 0eeddbaed81b4419ba946a691b5e158c P bccd627ea4c349d19e0937a0d23f52eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:19:32.089771 11727 ts_tablet_manager.cc:1905] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:19:32.089962 11728 log.cc:1196] T 0eeddbaed81b4419ba946a691b5e158c P bccd627ea4c349d19e0937a0d23f52eb: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-0/wal/wals/0eeddbaed81b4419ba946a691b5e158c
I20260807 18:19:32.092363 11729 ts_tablet_manager.cc:1905] T 0eeddbaed81b4419ba946a691b5e158c P 7fdb0a8077034b7cb1d3542e7b6f3acb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:19:32.093744 11728 ts_tablet_manager.cc:1939] T 0eeddbaed81b4419ba946a691b5e158c P bccd627ea4c349d19e0937a0d23f52eb: Deleting consensus metadata
I20260807 18:19:32.096864 10963 catalog_manager.cc:4841] TS bccd627ea4c349d19e0937a0d23f52eb (127.17.110.1:36441): tablet 0eeddbaed81b4419ba946a691b5e158c (table test-table [id=b18dfa5e4ca34551bcd27e494e63abbf]) successfully deleted
I20260807 18:19:32.098073 11683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:32.084103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49516 (local address 127.17.110.1:36441)
0807 18:19:32.084320 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:32.084326 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:32.087477 (+ 3151us) server_negotiation.cc:407] Connection header received
0807 18:19:32.087814 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:32.087820 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:32.087976 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:32.088072 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:32.088836 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:32.090080 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:32.092422 (+ 2342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:32.093166 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:32.093317 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:32.095239 (+ 1922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:32.095254 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:32.095272 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:19:32.096210 (+ 938us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:32.096304 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:19:32.096334 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:32.096438 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:32.097490 (+ 1052us) server_negotiation.cc:299] Negotiation successful
0807 18:19:32.097645 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:19:32.099469 11245 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:19:32.103197 11729 ts_tablet_manager.cc:1918] T 0eeddbaed81b4419ba946a691b5e158c P 7fdb0a8077034b7cb1d3542e7b6f3acb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:19:32.103404 11729 log.cc:1196] T 0eeddbaed81b4419ba946a691b5e158c P 7fdb0a8077034b7cb1d3542e7b6f3acb: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-2/wal/wals/0eeddbaed81b4419ba946a691b5e158c
I20260807 18:19:32.103967 11729 ts_tablet_manager.cc:1939] T 0eeddbaed81b4419ba946a691b5e158c P 7fdb0a8077034b7cb1d3542e7b6f3acb: Deleting consensus metadata
I20260807 18:19:32.103940 11727 ts_tablet_manager.cc:1918] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:19:32.104128 11727 log.cc:1196] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126429568977-17848-0/minicluster-data/ts-1/wal/wals/0eeddbaed81b4419ba946a691b5e158c
I20260807 18:19:32.105942 11727 ts_tablet_manager.cc:1939] T 0eeddbaed81b4419ba946a691b5e158c P 33d215770177400ba9ab2dfe06647979: Deleting consensus metadata
I20260807 18:19:32.107288 10965 catalog_manager.cc:4841] TS 7fdb0a8077034b7cb1d3542e7b6f3acb (127.17.110.3:44547): tablet 0eeddbaed81b4419ba946a691b5e158c (table test-table [id=b18dfa5e4ca34551bcd27e494e63abbf]) successfully deleted
I20260807 18:19:32.108094 10962 catalog_manager.cc:4841] TS 33d215770177400ba9ab2dfe06647979 (127.17.110.2:36199): tablet 0eeddbaed81b4419ba946a691b5e158c (table test-table [id=b18dfa5e4ca34551bcd27e494e63abbf]) successfully deleted
W20260807 18:19:32.109496 10962 catalog_manager.cc:4808] TS 33d215770177400ba9ab2dfe06647979 (127.17.110.2:36199): delete failed for tablet 0eeddbaed81b4419ba946a691b5e158c because the tablet was not found. No further retry: Not found: Tablet not found: 0eeddbaed81b4419ba946a691b5e158c
I20260807 18:19:32.207405 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 11068
I20260807 18:19:32.232765 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 11260
I20260807 18:19:32.240906 11686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:32.233650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48434 (local address 127.17.110.2:36199)
0807 18:19:32.233837 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:32.233845 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:32.233923 (+ 78us) server_negotiation.cc:407] Connection header received
0807 18:19:32.234165 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:32.234169 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:32.234260 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:32.234393 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:32.234926 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:32.235771 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:32.237162 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:32.237738 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:32.237896 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:32.239344 (+ 1448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:32.239354 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:32.239371 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:19:32.240047 (+ 676us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:32.240103 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:19:32.240126 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:32.240284 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:32.240427 (+ 143us) server_negotiation.cc:299] Negotiation successful
0807 18:19:32.240543 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 18:19:32.241791 11450 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:19:32.360257 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 11260
I20260807 18:19:32.393218 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 11460
I20260807 18:19:32.400938 11685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:32.394048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41984 (local address 127.17.110.3:44547)
0807 18:19:32.394230 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:32.394234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:32.394254 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:19:32.394513 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:32.394518 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:32.394662 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:32.394838 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:32.395242 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:32.396007 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:32.397546 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:32.397959 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:32.398160 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:32.399405 (+ 1245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:32.399416 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:32.399434 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:19:32.400104 (+ 670us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:32.400164 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:19:32.400187 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:32.400293 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:32.400518 (+ 225us) server_negotiation.cc:299] Negotiation successful
0807 18:19:32.400640 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 18:19:32.401789 11653 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:19:32.495992 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 11460
I20260807 18:19:32.519028 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 10942
I20260807 18:19:32.520334 11031 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:19:32.633595 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 10942
2026-08-07T18:19:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3312 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:19:32 dist-test-slave-39l8 krb5kdc[11739](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:19:32 dist-test-slave-39l8 krb5kdc[11739](info): set up 2 sockets
Aug 07 18:19:32 dist-test-slave-39l8 krb5kdc[11739](info): commencing operation
krb5kdc: starting...
W20260807 18:19:33.983376 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.005s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:19:34 dist-test-slave-39l8 krb5kdc[11739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:19:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:19:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.110.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
I20260807 18:19:34.351091 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:41903
--webserver_interface=127.17.110.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:42249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:41903
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:19:34.749073 11752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:19:34.749490 11752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:19:34.749666 11752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:19:34.757547 11752 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:19:34.757789 11752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:19:34.757910 11752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:19:34.758021 11752 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:19:34.758109 11752 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:19:34.769613 11752 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:42249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:41903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:41903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:19:34.771447 11752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:19:34.772964 11752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:19:34.787537 11763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:34.787492 11759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:34.787693 11765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:19:34.788580 11752 server_base.cc:1034] running on GCE node
I20260807 18:19:34.789577 11752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:19:34.791373 11752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:19:34.792680 11752 hybrid_clock.cc:648] HybridClock initialized: now 1786126774792612 us; error 53 us; skew 500 ppm
I20260807 18:19:34.793145 11752 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:19:34 dist-test-slave-39l8 krb5kdc[11739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126774, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:19:34.797171 11752 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:19:34.799057 11752 webserver.cc:460] Webserver started at http://127.17.110.62:35977/ using document root <none> and password file <none>
I20260807 18:19:34.799911 11752 fs_manager.cc:359] Metadata directory not provided
I20260807 18:19:34.800083 11752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:19:34.800453 11752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:19:34.804075 11752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "9e47390e368a45fa85a204d30b48e978"
format_stamp: "Formatted at 2026-08-07 18:19:34 on dist-test-slave-39l8"
server_key: "6e887049a8e56c03c26044eb59e72bb6"
server_key_iv: "d04555bb50bd6d10cba4961ee2d10f9d"
server_key_version: "encryptionkey@0"
I20260807 18:19:34.805064 11752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "9e47390e368a45fa85a204d30b48e978"
format_stamp: "Formatted at 2026-08-07 18:19:34 on dist-test-slave-39l8"
server_key: "6e887049a8e56c03c26044eb59e72bb6"
server_key_iv: "d04555bb50bd6d10cba4961ee2d10f9d"
server_key_version: "encryptionkey@0"
I20260807 18:19:34.812496 11752 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
I20260807 18:19:34.817253 11774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:34.819263 11752 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:19:34.819548 11752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "9e47390e368a45fa85a204d30b48e978"
format_stamp: "Formatted at 2026-08-07 18:19:34 on dist-test-slave-39l8"
server_key: "6e887049a8e56c03c26044eb59e72bb6"
server_key_iv: "d04555bb50bd6d10cba4961ee2d10f9d"
server_key_version: "encryptionkey@0"
I20260807 18:19:34.819890 11752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:19:34.850278 11752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:19:34.855736 11752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:19:34.856145 11752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:19:34.879832 11752 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:41903
I20260807 18:19:34.879907 11852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:41903 every 8 connection(s)
I20260807 18:19:34.881505 11752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:19:34.885957 11855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:19:34.889307 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 11752
I20260807 18:19:34.889551 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:19:34.890035 17848 external_mini_cluster.cc:1428] Setting key 44a25a6382cf4629e84a6ec173cd019c
I20260807 18:19:34.901439 11855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e47390e368a45fa85a204d30b48e978: Bootstrap starting.
I20260807 18:19:34.906018 11855 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e47390e368a45fa85a204d30b48e978: Neither blocks nor log segments found. Creating new log.
I20260807 18:19:34.907666 11855 log.cc:824] T 00000000000000000000000000000000 P 9e47390e368a45fa85a204d30b48e978: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:19:34 dist-test-slave-39l8 krb5kdc[11739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:19:34.911933 11855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e47390e368a45fa85a204d30b48e978: No bootstrap required, opened a new log
I20260807 18:19:34.918969 11858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:34.892633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36242 (local address 127.17.110.62:41903)
0807 18:19:34.893613 (+ 980us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:34.893626 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:34.893683 (+ 57us) server_negotiation.cc:407] Connection header received
0807 18:19:34.895798 (+ 2115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:34.895832 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:34.896387 (+ 555us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:34.896822 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:34.899451 (+ 2629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:34.900982 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:34.903110 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:34.903931 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:34.904105 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:34.910023 (+ 5918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:34.910040 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:34.910059 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:19:34.910107 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:34.913756 (+ 3649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:34.914797 (+ 1041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:34.914810 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:34.914819 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:34.915058 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:34.915571 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:34.915575 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:34.915577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:34.916272 (+ 695us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:34.916644 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:34.917216 (+ 572us) server_negotiation.cc:299] Negotiation successful
0807 18:19:34.917722 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":724,"thread_start_us":471,"threads_started":1}
I20260807 18:19:34.921417 11855 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e47390e368a45fa85a204d30b48e978 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e47390e368a45fa85a204d30b48e978" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 41903 } }
I20260807 18:19:34.921909 11855 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e47390e368a45fa85a204d30b48e978 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:19:34.922065 11855 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e47390e368a45fa85a204d30b48e978 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e47390e368a45fa85a204d30b48e978, State: Initialized, Role: FOLLOWER
I20260807 18:19:34.922824 11855 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e47390e368a45fa85a204d30b48e978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e47390e368a45fa85a204d30b48e978" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 41903 } }
I20260807 18:19:34.923156 11855 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e47390e368a45fa85a204d30b48e978 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:19:34.923316 11855 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e47390e368a45fa85a204d30b48e978 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:19:34.923514 11855 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e47390e368a45fa85a204d30b48e978 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:19:34.926864 11855 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e47390e368a45fa85a204d30b48e978 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e47390e368a45fa85a204d30b48e978" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 41903 } }
I20260807 18:19:34.927615 11855 leader_election.cc:302] T 00000000000000000000000000000000 P 9e47390e368a45fa85a204d30b48e978 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e47390e368a45fa85a204d30b48e978; no voters:
I20260807 18:19:34.928395 11855 leader_election.cc:288] T 00000000000000000000000000000000 P 9e47390e368a45fa85a204d30b48e978 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:19:34.928577 11861 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e47390e368a45fa85a204d30b48e978 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:19:34.929479 11861 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e47390e368a45fa85a204d30b48e978 [term 1 LEADER]: Becoming Leader. State: Replica: 9e47390e368a45fa85a204d30b48e978, State: Running, Role: LEADER
I20260807 18:19:34.930137 11861 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e47390e368a45fa85a204d30b48e978 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e47390e368a45fa85a204d30b48e978" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 41903 } }
I20260807 18:19:34.930994 11855 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e47390e368a45fa85a204d30b48e978 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:19:34.933578 11866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e47390e368a45fa85a204d30b48e978 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e47390e368a45fa85a204d30b48e978. Latest consensus state: current_term: 1 leader_uuid: "9e47390e368a45fa85a204d30b48e978" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e47390e368a45fa85a204d30b48e978" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 41903 } } }
I20260807 18:19:34.934051 11866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e47390e368a45fa85a204d30b48e978 [sys.catalog]: This master's current role is: LEADER
I20260807 18:19:34.937611 11864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e47390e368a45fa85a204d30b48e978 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e47390e368a45fa85a204d30b48e978" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e47390e368a45fa85a204d30b48e978" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 41903 } } }
I20260807 18:19:34.938210 11873 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:19:34.944972 11873 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:19:34.938202 11864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e47390e368a45fa85a204d30b48e978 [sys.catalog]: This master's current role is: LEADER
I20260807 18:19:34.956077 11873 catalog_manager.cc:1347] Generated new cluster ID: 41ab6dea63a641929df4bdae055f91a6
I20260807 18:19:34.956341 11873 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:19:34.974246 11873 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:19:34.975867 11873 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:19:34.986343 11873 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e47390e368a45fa85a204d30b48e978: Generated new TSK 0
I20260807 18:19:34.987442 11873 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.110.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
I20260807 18:19:35.080588 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.1:0
--local_ip_for_outbound_sockets=127.17.110.1
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:41903
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:42249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:19:35.487653 11894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:19:35.487978 11894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:19:35.488144 11894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:19:35.496631 11894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:19:35.496778 11894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:19:35.497008 11894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.1
I20260807 18:19:35.507886 11894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:42249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:41903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.1
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:19:35.509373 11894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:19:35.511016 11894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:19:35.526921 11901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:35.526829 11903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:35.526821 11906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:19:35.533502 11894 server_base.cc:1034] running on GCE node
I20260807 18:19:35.534287 11894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:19:35.535539 11894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:19:35.537263 11894 hybrid_clock.cc:648] HybridClock initialized: now 1786126775537184 us; error 48 us; skew 500 ppm
I20260807 18:19:35.537945 11894 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:19:35 dist-test-slave-39l8 krb5kdc[11739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126775, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:19:35.544426 11894 init.cc:377] Logged in from keytab as kudu/127.17.110.1@KRBTEST.COM (short username kudu)
I20260807 18:19:35.548799 11894 webserver.cc:460] Webserver started at http://127.17.110.1:37497/ using document root <none> and password file <none>
I20260807 18:19:35.550092 11894 fs_manager.cc:359] Metadata directory not provided
I20260807 18:19:35.550298 11894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:19:35.550740 11894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:19:35.554283 11894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-0/data/instance:
uuid: "16e21272f643412cada52b5893cae14a"
format_stamp: "Formatted at 2026-08-07 18:19:35 on dist-test-slave-39l8"
server_key: "0891aa35536a355fc05df3c217e179b7"
server_key_iv: "216ae552941a617ab6326273f3ef6a87"
server_key_version: "encryptionkey@0"
I20260807 18:19:35.555335 11894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance:
uuid: "16e21272f643412cada52b5893cae14a"
format_stamp: "Formatted at 2026-08-07 18:19:35 on dist-test-slave-39l8"
server_key: "0891aa35536a355fc05df3c217e179b7"
server_key_iv: "216ae552941a617ab6326273f3ef6a87"
server_key_version: "encryptionkey@0"
I20260807 18:19:35.561777 11894 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260807 18:19:35.566146 11916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:35.568274 11894 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:19:35.568534 11894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-0/wal
uuid: "16e21272f643412cada52b5893cae14a"
format_stamp: "Formatted at 2026-08-07 18:19:35 on dist-test-slave-39l8"
server_key: "0891aa35536a355fc05df3c217e179b7"
server_key_iv: "216ae552941a617ab6326273f3ef6a87"
server_key_version: "encryptionkey@0"
I20260807 18:19:35.568810 11894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:19:35.596143 11894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:19:35.601528 11894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:19:35.602022 11894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:19:35.603266 11894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:19:35.605000 11894 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:19:35.605132 11894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:35.605319 11894 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:19:35.605398 11894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:35.647409 11894 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.1:45755
I20260807 18:19:35.647519 12084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.1:45755 every 8 connection(s)
I20260807 18:19:35.648887 11894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
I20260807 18:19:35.650450 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 11894
I20260807 18:19:35.650679 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance
I20260807 18:19:35.651290 17848 external_mini_cluster.cc:1428] Setting key 22bb801f79401f75ea77d9e83dcb539d
Aug 07 18:19:35 dist-test-slave-39l8 krb5kdc[11739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126775, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:19:35.678920 12090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:35.652295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:40643 (local address 127.17.110.62:41903)
0807 18:19:35.652832 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:35.652840 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:35.654215 (+ 1375us) server_negotiation.cc:407] Connection header received
0807 18:19:35.656887 (+ 2672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:35.656897 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:35.657021 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:35.657169 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:35.660229 (+ 3060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:35.662087 (+ 1858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:35.664351 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:35.665117 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:35.665329 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:35.669671 (+ 4342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:35.669683 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:35.669690 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:19:35.669745 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:35.674262 (+ 4517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:35.675454 (+ 1192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:35.675464 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:35.675470 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:35.675667 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:35.676308 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:35.676317 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:35.676324 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:35.676807 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:35.676993 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:35.678028 (+ 1035us) server_negotiation.cc:299] Negotiation successful
0807 18:19:35.678380 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":314,"threads_started":1}
I20260807 18:19:35.679992 12091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:35.653002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:41903 (local address 127.17.110.1:40643)
0807 18:19:35.654009 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:35.654061 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:35.656497 (+ 2436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:35.657886 (+ 1389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:35.657899 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:35.658639 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:19:35.659768 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:35.659803 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:35.662435 (+ 2632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:35.662441 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:35.664101 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:35.664125 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:35.665444 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:35.665450 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:35.665719 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:35.666589 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:19:35.666616 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:19:35.669279 (+ 2663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:19:35.674493 (+ 5214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:19:35.674504 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:19:35.674521 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:19:35.675213 (+ 692us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:19:35.675833 (+ 620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:19:35.675838 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:19:35.675840 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:19:35.676141 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:19:35.677040 (+ 899us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:19:35.677049 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:19:35.677501 (+ 452us) client_negotiation.cc:769] Sending connection context
0807 18:19:35.677817 (+ 316us) client_negotiation.cc:240] Negotiation successful
0807 18:19:35.678381 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"thread_start_us":410,"threads_started":1}
I20260807 18:19:35.682195 12085 heartbeater.cc:347] Connected to a master server at 127.17.110.62:41903
I20260807 18:19:35.682572 12085 heartbeater.cc:464] Registering TS with master...
I20260807 18:19:35.683498 12085 heartbeater.cc:511] Master 127.17.110.62:41903 requested a full tablet report, sending...
I20260807 18:19:35.686412 11796 ts_manager.cc:194] Registered new tserver with Master: 16e21272f643412cada52b5893cae14a (127.17.110.1:45755)
I20260807 18:19:35.688300 11796 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.1@KRBTEST.COM'} at 127.17.110.1:40643
WARNING: no policy specified for kudu/127.17.110.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
I20260807 18:19:35.735886 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.2:0
--local_ip_for_outbound_sockets=127.17.110.2
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:41903
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:42249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:19:36.131287 12095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:19:36.131666 12095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:19:36.131831 12095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:19:36.139986 12095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:19:36.140126 12095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:19:36.140323 12095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.2
I20260807 18:19:36.151448 12095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:42249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:41903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.2
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:19:36.152931 12095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:19:36.154413 12095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:19:36.170818 12108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:36.170867 12111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:36.170923 12106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:19:36.172297 12095 server_base.cc:1034] running on GCE node
I20260807 18:19:36.173193 12095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:19:36.174631 12095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:19:36.176060 12095 hybrid_clock.cc:648] HybridClock initialized: now 1786126776175926 us; error 105 us; skew 500 ppm
I20260807 18:19:36.176635 12095 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:19:36 dist-test-slave-39l8 krb5kdc[11739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126776, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:19:36.181025 12095 init.cc:377] Logged in from keytab as kudu/127.17.110.2@KRBTEST.COM (short username kudu)
I20260807 18:19:36.182916 12095 webserver.cc:460] Webserver started at http://127.17.110.2:41483/ using document root <none> and password file <none>
I20260807 18:19:36.183887 12095 fs_manager.cc:359] Metadata directory not provided
I20260807 18:19:36.184054 12095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:19:36.184420 12095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:19:36.187979 12095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-1/data/instance:
uuid: "84f83b8cfb234aa5a779d69106186dea"
format_stamp: "Formatted at 2026-08-07 18:19:36 on dist-test-slave-39l8"
server_key: "ec9167a32cdde2b718eaca849e8f5b94"
server_key_iv: "aed2e6104618069463b570b719568a3e"
server_key_version: "encryptionkey@0"
I20260807 18:19:36.189386 12095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance:
uuid: "84f83b8cfb234aa5a779d69106186dea"
format_stamp: "Formatted at 2026-08-07 18:19:36 on dist-test-slave-39l8"
server_key: "ec9167a32cdde2b718eaca849e8f5b94"
server_key_iv: "aed2e6104618069463b570b719568a3e"
server_key_version: "encryptionkey@0"
I20260807 18:19:36.196408 12095 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260807 18:19:36.201758 12123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:36.203727 12095 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 18:19:36.203971 12095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-1/wal
uuid: "84f83b8cfb234aa5a779d69106186dea"
format_stamp: "Formatted at 2026-08-07 18:19:36 on dist-test-slave-39l8"
server_key: "ec9167a32cdde2b718eaca849e8f5b94"
server_key_iv: "aed2e6104618069463b570b719568a3e"
server_key_version: "encryptionkey@0"
I20260807 18:19:36.204253 12095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:19:36.230629 12095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:19:36.236755 12095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:19:36.237178 12095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:19:36.238552 12095 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:19:36.240325 12095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:19:36.240417 12095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:36.240571 12095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:19:36.240639 12095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:36.282094 12095 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.2:45081
I20260807 18:19:36.282160 12292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.2:45081 every 8 connection(s)
I20260807 18:19:36.284008 12095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
I20260807 18:19:36.293418 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 12095
I20260807 18:19:36.293711 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance
I20260807 18:19:36.294445 17848 external_mini_cluster.cc:1428] Setting key c6bb4d8906f7c89d32c0e0aeb4a571be
Aug 07 18:19:36 dist-test-slave-39l8 krb5kdc[11739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126776, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:19:36.307328 12296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:36.287680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:50527 (local address 127.17.110.62:41903)
0807 18:19:36.288260 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:36.288265 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:36.289114 (+ 849us) server_negotiation.cc:407] Connection header received
0807 18:19:36.291251 (+ 2137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:36.291258 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:36.291376 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:36.291503 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:36.293292 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:36.294187 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:36.296173 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:36.296808 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:36.296962 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:36.300772 (+ 3810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:36.300778 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:36.300781 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:19:36.300816 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:36.303738 (+ 2922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:36.304650 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:36.304654 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:36.304657 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:36.304794 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:36.305293 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:36.305298 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:36.305300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:36.305647 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:36.305820 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:36.306632 (+ 812us) server_negotiation.cc:299] Negotiation successful
0807 18:19:36.306942 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":308,"threads_started":1}
I20260807 18:19:36.308410 12297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:36.288020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:41903 (local address 127.17.110.2:50527)
0807 18:19:36.288908 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:36.288962 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:36.290891 (+ 1929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:36.291791 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:36.291802 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:36.292435 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:19:36.293071 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:36.293100 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:36.294371 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:36.294375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:36.295982 (+ 1607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:36.295997 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:36.297064 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:36.297072 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:36.297329 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:36.298073 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:19:36.298104 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:19:36.300496 (+ 2392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:19:36.303925 (+ 3429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:19:36.303938 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:19:36.303959 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:19:36.304466 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:19:36.304918 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:19:36.304923 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:19:36.304926 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:19:36.305152 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:19:36.305789 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:19:36.305798 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:19:36.306207 (+ 409us) client_negotiation.cc:769] Sending connection context
0807 18:19:36.306500 (+ 293us) client_negotiation.cc:240] Negotiation successful
0807 18:19:36.307136 (+ 636us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":380,"threads_started":1}
I20260807 18:19:36.310287 12294 heartbeater.cc:347] Connected to a master server at 127.17.110.62:41903
I20260807 18:19:36.310636 12294 heartbeater.cc:464] Registering TS with master...
I20260807 18:19:36.311447 12294 heartbeater.cc:511] Master 127.17.110.62:41903 requested a full tablet report, sending...
I20260807 18:19:36.313426 11796 ts_manager.cc:194] Registered new tserver with Master: 84f83b8cfb234aa5a779d69106186dea (127.17.110.2:45081)
I20260807 18:19:36.314909 11796 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.2@KRBTEST.COM'} at 127.17.110.2:50527
WARNING: no policy specified for kudu/127.17.110.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
I20260807 18:19:36.377151 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.3:0
--local_ip_for_outbound_sockets=127.17.110.3
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:41903
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:42249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:19:36.692898 12085 heartbeater.cc:503] Master 127.17.110.62:41903 was elected leader, sending a full tablet report...
W20260807 18:19:36.763091 12300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:19:36.763434 12300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:19:36.763537 12300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:19:36.771512 12300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:19:36.771680 12300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:19:36.771806 12300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.3
I20260807 18:19:36.786233 12300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:42249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:41903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.3
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:19:36.787801 12300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:19:36.789321 12300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:19:36.805755 12313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:36.805882 12315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:36.805883 12310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:19:36.806826 12300 server_base.cc:1034] running on GCE node
I20260807 18:19:36.807636 12300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:19:36.808990 12300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:19:36.810263 12300 hybrid_clock.cc:648] HybridClock initialized: now 1786126776810206 us; error 60 us; skew 500 ppm
I20260807 18:19:36.810688 12300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:19:36 dist-test-slave-39l8 krb5kdc[11739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126776, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:19:36.820111 12300 init.cc:377] Logged in from keytab as kudu/127.17.110.3@KRBTEST.COM (short username kudu)
I20260807 18:19:36.821887 12300 webserver.cc:460] Webserver started at http://127.17.110.3:34089/ using document root <none> and password file <none>
I20260807 18:19:36.822675 12300 fs_manager.cc:359] Metadata directory not provided
I20260807 18:19:36.822885 12300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:19:36.823218 12300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:19:36.827574 12300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-2/data/instance:
uuid: "be50fccdbbb84336aac97309330b9050"
format_stamp: "Formatted at 2026-08-07 18:19:36 on dist-test-slave-39l8"
server_key: "ea9ffb40dc099201f11db5ee8bb74723"
server_key_iv: "0884fa2c9297e54cda612382e943b8c5"
server_key_version: "encryptionkey@0"
I20260807 18:19:36.828589 12300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance:
uuid: "be50fccdbbb84336aac97309330b9050"
format_stamp: "Formatted at 2026-08-07 18:19:36 on dist-test-slave-39l8"
server_key: "ea9ffb40dc099201f11db5ee8bb74723"
server_key_iv: "0884fa2c9297e54cda612382e943b8c5"
server_key_version: "encryptionkey@0"
I20260807 18:19:36.835337 12300 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 18:19:36.840158 12326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:36.842060 12300 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:19:36.842286 12300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-2/wal
uuid: "be50fccdbbb84336aac97309330b9050"
format_stamp: "Formatted at 2026-08-07 18:19:36 on dist-test-slave-39l8"
server_key: "ea9ffb40dc099201f11db5ee8bb74723"
server_key_iv: "0884fa2c9297e54cda612382e943b8c5"
server_key_version: "encryptionkey@0"
I20260807 18:19:36.842517 12300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:19:36.869434 12300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:19:36.875303 12300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:19:36.875628 12300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:19:36.876821 12300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:19:36.878571 12300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:19:36.878731 12300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:36.878835 12300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:19:36.878882 12300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:36.920162 12300 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.3:44211
I20260807 18:19:36.920243 12486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.3:44211 every 8 connection(s)
I20260807 18:19:36.921880 12300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
I20260807 18:19:36.926285 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 12300
I20260807 18:19:36.926537 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance
I20260807 18:19:36.927068 17848 external_mini_cluster.cc:1428] Setting key c0b5d16af623b82bdb379fc4a19d6d09
Aug 07 18:19:36 dist-test-slave-39l8 krb5kdc[11739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126776, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:19:36.949213 12492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:36.925248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:42525 (local address 127.17.110.62:41903)
0807 18:19:36.925838 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:36.925842 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:36.927261 (+ 1419us) server_negotiation.cc:407] Connection header received
0807 18:19:36.929919 (+ 2658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:36.929924 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:36.930031 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:36.930151 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:36.932749 (+ 2598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:36.933786 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:36.935820 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:36.936308 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:36.936437 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:36.940720 (+ 4283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:36.940728 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:36.940733 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:19:36.940776 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:36.944997 (+ 4221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:36.946208 (+ 1211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:36.946213 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:36.946215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:36.946360 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:36.947058 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:36.947063 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:36.947065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:36.947411 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:36.947555 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:36.948669 (+ 1114us) server_negotiation.cc:299] Negotiation successful
0807 18:19:36.948882 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":337,"threads_started":1}
I20260807 18:19:36.950311 12493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:36.925830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:41903 (local address 127.17.110.3:42525)
0807 18:19:36.926878 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:36.926943 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:36.929486 (+ 2543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:36.930633 (+ 1147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:36.930647 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:36.931680 (+ 1033us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:19:36.932516 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:36.932542 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:36.934009 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:36.934014 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:36.935619 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:36.935637 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:36.936556 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:36.936561 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:36.936840 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:36.937637 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:19:36.937705 (+ 68us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:19:36.940389 (+ 2684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:19:36.945258 (+ 4869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:19:36.945272 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:19:36.945300 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:19:36.946014 (+ 714us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:19:36.946495 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:19:36.946499 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:19:36.946502 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:19:36.946874 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:19:36.947596 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:19:36.947603 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:19:36.948215 (+ 612us) client_negotiation.cc:769] Sending connection context
0807 18:19:36.948495 (+ 280us) client_negotiation.cc:240] Negotiation successful
0807 18:19:36.948956 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":455,"threads_started":1}
I20260807 18:19:36.952445 12487 heartbeater.cc:347] Connected to a master server at 127.17.110.62:41903
I20260807 18:19:36.952898 12487 heartbeater.cc:464] Registering TS with master...
I20260807 18:19:36.953923 12487 heartbeater.cc:511] Master 127.17.110.62:41903 requested a full tablet report, sending...
I20260807 18:19:36.956069 11796 ts_manager.cc:194] Registered new tserver with Master: be50fccdbbb84336aac97309330b9050 (127.17.110.3:44211)
I20260807 18:19:36.957132 11796 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.3@KRBTEST.COM'} at 127.17.110.3:42525
I20260807 18:19:36.959247 17848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:19:36.979738 12492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:36.963915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36256 (local address 127.17.110.62:41903)
0807 18:19:36.964139 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:36.964148 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:36.964634 (+ 486us) server_negotiation.cc:407] Connection header received
0807 18:19:36.964879 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:36.964884 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:36.964958 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:36.965077 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:36.966659 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:36.967491 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:36.969147 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:36.969664 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:36.969825 (+ 161us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:36.973282 (+ 3457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:36.973289 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:36.973292 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:19:36.973343 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:36.976313 (+ 2970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:36.977322 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:36.977327 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:36.977330 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:36.977479 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:36.978030 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:36.978034 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:36.978036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:36.978392 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:36.978756 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:36.979174 (+ 418us) server_negotiation.cc:299] Negotiation successful
0807 18:19:36.979422 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 18:19:36.984807 11796 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36256:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:19:36.992458 11796 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:19:37.026897 12506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:37.007277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:44211 (local address 127.0.0.1:42026)
0807 18:19:37.008774 (+ 1497us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:37.008840 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:37.009102 (+ 262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:37.011128 (+ 2026us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:37.011136 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:37.011173 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:19:37.012577 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:37.012597 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:37.017057 (+ 4460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:37.017066 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:37.021902 (+ 4836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:37.021931 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:37.025468 (+ 3537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:37.025481 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:37.025774 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:37.025835 (+ 61us) client_negotiation.cc:769] Sending connection context
0807 18:19:37.025968 (+ 133us) client_negotiation.cc:240] Negotiation successful
0807 18:19:37.026223 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"mutex_wait_us":96,"thread_start_us":470,"threads_started":1}
I20260807 18:19:37.028676 12507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:37.007616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42026 (local address 127.17.110.3:44211)
0807 18:19:37.009448 (+ 1832us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:37.009458 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:37.009497 (+ 39us) server_negotiation.cc:407] Connection header received
0807 18:19:37.009613 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:37.009675 (+ 62us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:37.010093 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:37.010349 (+ 256us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:19:37.013137 (+ 2788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:37.016799 (+ 3662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:37.022316 (+ 5517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:37.024637 (+ 2321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:37.024870 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:37.025245 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:37.027732 (+ 2487us) server_negotiation.cc:299] Negotiation successful
0807 18:19:37.028024 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1488,"thread_start_us":1244,"threads_started":1}
I20260807 18:19:37.029584 12511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:37.009576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:45081 (local address 127.0.0.1:55816)
0807 18:19:37.010228 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:37.010249 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:37.010449 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:37.013846 (+ 3397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:37.013856 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:37.013903 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:19:37.014533 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:37.014551 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:37.017549 (+ 2998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:37.017555 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:37.021900 (+ 4345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:37.021931 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:37.025159 (+ 3228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:37.025171 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:37.025568 (+ 397us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:37.025669 (+ 101us) client_negotiation.cc:769] Sending connection context
0807 18:19:37.025892 (+ 223us) client_negotiation.cc:240] Negotiation successful
0807 18:19:37.026228 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"mutex_wait_us":6,"thread_start_us":322,"threads_started":1}
I20260807 18:19:37.031075 12513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:37.009715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55816 (local address 127.17.110.2:45081)
0807 18:19:37.011048 (+ 1333us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:37.011056 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:37.011224 (+ 168us) server_negotiation.cc:407] Connection header received
0807 18:19:37.011344 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:37.011352 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:37.011822 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:37.015013 (+ 3191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:19:37.015369 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:37.017326 (+ 1957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:37.022367 (+ 5041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:37.024801 (+ 2434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:37.029919 (+ 5118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:37.030278 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:37.030380 (+ 102us) server_negotiation.cc:299] Negotiation successful
0807 18:19:37.030619 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1053,"thread_start_us":896,"threads_started":1}
I20260807 18:19:37.036923 12187 tablet_service.cc:1467] Processing CreateTablet for tablet e212944892ef4b6d9a8b7ea343cf83c7 (DEFAULT_TABLE table=test-table [id=9cd5d3c34d17468d9f88fdb890ab880c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:19:37.038022 12392 tablet_service.cc:1467] Processing CreateTablet for tablet e212944892ef4b6d9a8b7ea343cf83c7 (DEFAULT_TABLE table=test-table [id=9cd5d3c34d17468d9f88fdb890ab880c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:19:37.038928 12187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e212944892ef4b6d9a8b7ea343cf83c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:19:37.039642 12506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:37.011335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:45755 (local address 127.0.0.1:34390)
0807 18:19:37.027255 (+ 15920us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:37.027283 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:37.027463 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:37.031424 (+ 3961us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:37.031430 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:37.031467 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:19:37.031697 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:37.031714 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:37.034126 (+ 2412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:37.034133 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:37.036166 (+ 2033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:37.036192 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:37.038621 (+ 2429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:37.038629 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:37.038881 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:37.038935 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 18:19:37.039029 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 18:19:37.039293 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15758,"thread_start_us":17969,"threads_started":1}
I20260807 18:19:37.040645 12392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e212944892ef4b6d9a8b7ea343cf83c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:19:37.040941 12519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:37.010729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34390 (local address 127.17.110.1:45755)
0807 18:19:37.030254 (+ 19525us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:37.030264 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:37.030297 (+ 33us) server_negotiation.cc:407] Connection header received
0807 18:19:37.030415 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:37.030423 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:37.030765 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:37.031035 (+ 270us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:19:37.031916 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:37.033872 (+ 1956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:37.036589 (+ 2717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:37.038306 (+ 1717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:37.039960 (+ 1654us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:37.040266 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:37.040370 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 18:19:37.040567 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19305,"thread_start_us":21139,"threads_started":1}
I20260807 18:19:37.046444 11982 tablet_service.cc:1467] Processing CreateTablet for tablet e212944892ef4b6d9a8b7ea343cf83c7 (DEFAULT_TABLE table=test-table [id=9cd5d3c34d17468d9f88fdb890ab880c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:19:37.051450 11982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e212944892ef4b6d9a8b7ea343cf83c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:19:37.058467 12522 tablet_bootstrap.cc:492] T e212944892ef4b6d9a8b7ea343cf83c7 P 84f83b8cfb234aa5a779d69106186dea: Bootstrap starting.
I20260807 18:19:37.060261 12523 tablet_bootstrap.cc:492] T e212944892ef4b6d9a8b7ea343cf83c7 P be50fccdbbb84336aac97309330b9050: Bootstrap starting.
I20260807 18:19:37.062928 12522 tablet_bootstrap.cc:654] T e212944892ef4b6d9a8b7ea343cf83c7 P 84f83b8cfb234aa5a779d69106186dea: Neither blocks nor log segments found. Creating new log.
I20260807 18:19:37.064841 12523 tablet_bootstrap.cc:654] T e212944892ef4b6d9a8b7ea343cf83c7 P be50fccdbbb84336aac97309330b9050: Neither blocks nor log segments found. Creating new log.
I20260807 18:19:37.064818 12522 log.cc:824] T e212944892ef4b6d9a8b7ea343cf83c7 P 84f83b8cfb234aa5a779d69106186dea: Log is configured to *not* fsync() on all Append() calls
I20260807 18:19:37.066468 12523 log.cc:824] T e212944892ef4b6d9a8b7ea343cf83c7 P be50fccdbbb84336aac97309330b9050: Log is configured to *not* fsync() on all Append() calls
I20260807 18:19:37.072357 12523 tablet_bootstrap.cc:492] T e212944892ef4b6d9a8b7ea343cf83c7 P be50fccdbbb84336aac97309330b9050: No bootstrap required, opened a new log
I20260807 18:19:37.072368 12522 tablet_bootstrap.cc:492] T e212944892ef4b6d9a8b7ea343cf83c7 P 84f83b8cfb234aa5a779d69106186dea: No bootstrap required, opened a new log
I20260807 18:19:37.072885 12522 ts_tablet_manager.cc:1397] T e212944892ef4b6d9a8b7ea343cf83c7 P 84f83b8cfb234aa5a779d69106186dea: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.004s
I20260807 18:19:37.073009 12523 ts_tablet_manager.cc:1397] T e212944892ef4b6d9a8b7ea343cf83c7 P be50fccdbbb84336aac97309330b9050: Time spent bootstrapping tablet: real 0.013s user 0.012s sys 0.000s
I20260807 18:19:37.080979 12522 raft_consensus.cc:348] T e212944892ef4b6d9a8b7ea343cf83c7 P 84f83b8cfb234aa5a779d69106186dea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be50fccdbbb84336aac97309330b9050" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44211 } } peers { permanent_uuid: "84f83b8cfb234aa5a779d69106186dea" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 45081 } } peers { permanent_uuid: "16e21272f643412cada52b5893cae14a" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 45755 } }
I20260807 18:19:37.081506 12522 raft_consensus.cc:374] T e212944892ef4b6d9a8b7ea343cf83c7 P 84f83b8cfb234aa5a779d69106186dea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:19:37.081615 12522 raft_consensus.cc:729] T e212944892ef4b6d9a8b7ea343cf83c7 P 84f83b8cfb234aa5a779d69106186dea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84f83b8cfb234aa5a779d69106186dea, State: Initialized, Role: FOLLOWER
I20260807 18:19:37.082510 12522 consensus_queue.cc:260] T e212944892ef4b6d9a8b7ea343cf83c7 P 84f83b8cfb234aa5a779d69106186dea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be50fccdbbb84336aac97309330b9050" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44211 } } peers { permanent_uuid: "84f83b8cfb234aa5a779d69106186dea" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 45081 } } peers { permanent_uuid: "16e21272f643412cada52b5893cae14a" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 45755 } }
I20260807 18:19:37.083145 12526 tablet_bootstrap.cc:492] T e212944892ef4b6d9a8b7ea343cf83c7 P 16e21272f643412cada52b5893cae14a: Bootstrap starting.
I20260807 18:19:37.087077 12294 heartbeater.cc:503] Master 127.17.110.62:41903 was elected leader, sending a full tablet report...
I20260807 18:19:37.086499 12523 raft_consensus.cc:348] T e212944892ef4b6d9a8b7ea343cf83c7 P be50fccdbbb84336aac97309330b9050 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be50fccdbbb84336aac97309330b9050" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44211 } } peers { permanent_uuid: "84f83b8cfb234aa5a779d69106186dea" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 45081 } } peers { permanent_uuid: "16e21272f643412cada52b5893cae14a" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 45755 } }
I20260807 18:19:37.087864 12523 raft_consensus.cc:374] T e212944892ef4b6d9a8b7ea343cf83c7 P be50fccdbbb84336aac97309330b9050 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:19:37.087986 12522 ts_tablet_manager.cc:1428] T e212944892ef4b6d9a8b7ea343cf83c7 P 84f83b8cfb234aa5a779d69106186dea: Time spent starting tablet: real 0.015s user 0.008s sys 0.008s
I20260807 18:19:37.088078 12523 raft_consensus.cc:729] T e212944892ef4b6d9a8b7ea343cf83c7 P be50fccdbbb84336aac97309330b9050 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be50fccdbbb84336aac97309330b9050, State: Initialized, Role: FOLLOWER
I20260807 18:19:37.088655 12526 tablet_bootstrap.cc:654] T e212944892ef4b6d9a8b7ea343cf83c7 P 16e21272f643412cada52b5893cae14a: Neither blocks nor log segments found. Creating new log.
I20260807 18:19:37.088958 12523 consensus_queue.cc:260] T e212944892ef4b6d9a8b7ea343cf83c7 P be50fccdbbb84336aac97309330b9050 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be50fccdbbb84336aac97309330b9050" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44211 } } peers { permanent_uuid: "84f83b8cfb234aa5a779d69106186dea" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 45081 } } peers { permanent_uuid: "16e21272f643412cada52b5893cae14a" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 45755 } }
I20260807 18:19:37.090214 12526 log.cc:824] T e212944892ef4b6d9a8b7ea343cf83c7 P 16e21272f643412cada52b5893cae14a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:19:37.091243 12487 heartbeater.cc:503] Master 127.17.110.62:41903 was elected leader, sending a full tablet report...
I20260807 18:19:37.092048 12523 ts_tablet_manager.cc:1428] T e212944892ef4b6d9a8b7ea343cf83c7 P be50fccdbbb84336aac97309330b9050: Time spent starting tablet: real 0.019s user 0.017s sys 0.000s
I20260807 18:19:37.096763 12526 tablet_bootstrap.cc:492] T e212944892ef4b6d9a8b7ea343cf83c7 P 16e21272f643412cada52b5893cae14a: No bootstrap required, opened a new log
I20260807 18:19:37.097177 12526 ts_tablet_manager.cc:1397] T e212944892ef4b6d9a8b7ea343cf83c7 P 16e21272f643412cada52b5893cae14a: Time spent bootstrapping tablet: real 0.015s user 0.011s sys 0.000s
I20260807 18:19:37.105512 12526 raft_consensus.cc:348] T e212944892ef4b6d9a8b7ea343cf83c7 P 16e21272f643412cada52b5893cae14a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be50fccdbbb84336aac97309330b9050" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44211 } } peers { permanent_uuid: "84f83b8cfb234aa5a779d69106186dea" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 45081 } } peers { permanent_uuid: "16e21272f643412cada52b5893cae14a" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 45755 } }
I20260807 18:19:37.105963 12526 raft_consensus.cc:374] T e212944892ef4b6d9a8b7ea343cf83c7 P 16e21272f643412cada52b5893cae14a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:19:37.106041 12526 raft_consensus.cc:729] T e212944892ef4b6d9a8b7ea343cf83c7 P 16e21272f643412cada52b5893cae14a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16e21272f643412cada52b5893cae14a, State: Initialized, Role: FOLLOWER
I20260807 18:19:37.106822 12526 consensus_queue.cc:260] T e212944892ef4b6d9a8b7ea343cf83c7 P 16e21272f643412cada52b5893cae14a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be50fccdbbb84336aac97309330b9050" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44211 } } peers { permanent_uuid: "84f83b8cfb234aa5a779d69106186dea" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 45081 } } peers { permanent_uuid: "16e21272f643412cada52b5893cae14a" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 45755 } }
I20260807 18:19:37.108702 12526 ts_tablet_manager.cc:1428] T e212944892ef4b6d9a8b7ea343cf83c7 P 16e21272f643412cada52b5893cae14a: Time spent starting tablet: real 0.011s user 0.010s sys 0.003s
W20260807 18:19:37.151973 12089 tablet.cc:2369] T e212944892ef4b6d9a8b7ea343cf83c7 P 16e21272f643412cada52b5893cae14a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:19:37.173910 12488 tablet.cc:2369] T e212944892ef4b6d9a8b7ea343cf83c7 P be50fccdbbb84336aac97309330b9050: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:19:37.286773 12295 tablet.cc:2369] T e212944892ef4b6d9a8b7ea343cf83c7 P 84f83b8cfb234aa5a779d69106186dea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:19:37.319082 12532 raft_consensus.cc:482] T e212944892ef4b6d9a8b7ea343cf83c7 P 16e21272f643412cada52b5893cae14a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:19:37.319522 12532 raft_consensus.cc:504] T e212944892ef4b6d9a8b7ea343cf83c7 P 16e21272f643412cada52b5893cae14a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be50fccdbbb84336aac97309330b9050" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44211 } } peers { permanent_uuid: "84f83b8cfb234aa5a779d69106186dea" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 45081 } } peers { permanent_uuid: "16e21272f643412cada52b5893cae14a" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 45755 } }
I20260807 18:19:37.321728 12532 leader_election.cc:288] T e212944892ef4b6d9a8b7ea343cf83c7 P 16e21272f643412cada52b5893cae14a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be50fccdbbb84336aac97309330b9050 (127.17.110.3:44211), 84f83b8cfb234aa5a779d69106186dea (127.17.110.2:45081)
I20260807 18:19:37.330123 12507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:37.322247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:38867 (local address 127.17.110.3:44211)
0807 18:19:37.322460 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:37.322465 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:37.323077 (+ 612us) server_negotiation.cc:407] Connection header received
0807 18:19:37.323362 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:37.323368 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:37.323459 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:37.323572 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:19:37.324387 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:37.325552 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:37.327936 (+ 2384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:37.328988 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:37.329136 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:37.329197 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:37.329664 (+ 467us) server_negotiation.cc:299] Negotiation successful
0807 18:19:37.329828 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 18:19:37.330163 12533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:37.322178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:44211 (local address 127.17.110.1:38867)
0807 18:19:37.322944 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:37.322966 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:37.323124 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:37.323754 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:37.323759 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:37.323784 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:19:37.324197 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:37.324212 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:37.325698 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:37.325704 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:37.327739 (+ 2035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:37.327757 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:37.329233 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:37.329241 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:37.329438 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:37.329476 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 18:19:37.329592 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 18:19:37.329725 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"mutex_wait_us":137,"thread_start_us":382,"threads_started":1}
I20260807 18:19:37.330473 12513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:37.323345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:37769 (local address 127.17.110.2:45081)
0807 18:19:37.323580 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:37.323585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:37.323923 (+ 338us) server_negotiation.cc:407] Connection header received
0807 18:19:37.324083 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:37.324091 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:37.324177 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:37.324290 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:19:37.325170 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:37.326107 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:37.328345 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:37.329333 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:37.329492 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:37.329580 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:37.330033 (+ 453us) server_negotiation.cc:299] Negotiation successful
0807 18:19:37.330219 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 18:19:37.330979 12534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:37.323124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:45081 (local address 127.17.110.1:37769)
0807 18:19:37.323801 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:37.323816 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:37.323938 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:37.324387 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:37.324393 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:37.324455 (+ 62us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:19:37.324940 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:37.324955 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:37.326278 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:37.326283 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:37.328161 (+ 1878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:37.328177 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:37.329613 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:37.329622 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:37.329830 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:37.329870 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 18:19:37.329977 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 18:19:37.330140 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":2,"thread_start_us":365,"threads_started":1}
I20260807 18:19:37.331768 12420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e212944892ef4b6d9a8b7ea343cf83c7" candidate_uuid: "16e21272f643412cada52b5893cae14a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be50fccdbbb84336aac97309330b9050" is_pre_election: true
I20260807 18:19:37.332226 12220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e212944892ef4b6d9a8b7ea343cf83c7" candidate_uuid: "16e21272f643412cada52b5893cae14a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84f83b8cfb234aa5a779d69106186dea" is_pre_election: true
I20260807 18:19:37.332427 12420 raft_consensus.cc:2454] T e212944892ef4b6d9a8b7ea343cf83c7 P be50fccdbbb84336aac97309330b9050 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16e21272f643412cada52b5893cae14a in term 0.
I20260807 18:19:37.332783 12220 raft_consensus.cc:2454] T e212944892ef4b6d9a8b7ea343cf83c7 P 84f83b8cfb234aa5a779d69106186dea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16e21272f643412cada52b5893cae14a in term 0.
I20260807 18:19:37.333415 11918 leader_election.cc:302] T e212944892ef4b6d9a8b7ea343cf83c7 P 16e21272f643412cada52b5893cae14a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16e21272f643412cada52b5893cae14a, be50fccdbbb84336aac97309330b9050; no voters:
I20260807 18:19:37.334049 12532 raft_consensus.cc:2781] T e212944892ef4b6d9a8b7ea343cf83c7 P 16e21272f643412cada52b5893cae14a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:19:37.334321 12532 raft_consensus.cc:482] T e212944892ef4b6d9a8b7ea343cf83c7 P 16e21272f643412cada52b5893cae14a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:19:37.334597 12532 raft_consensus.cc:3037] T e212944892ef4b6d9a8b7ea343cf83c7 P 16e21272f643412cada52b5893cae14a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:19:37.337241 12532 raft_consensus.cc:504] T e212944892ef4b6d9a8b7ea343cf83c7 P 16e21272f643412cada52b5893cae14a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be50fccdbbb84336aac97309330b9050" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44211 } } peers { permanent_uuid: "84f83b8cfb234aa5a779d69106186dea" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 45081 } } peers { permanent_uuid: "16e21272f643412cada52b5893cae14a" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 45755 } }
I20260807 18:19:37.338021 12532 leader_election.cc:288] T e212944892ef4b6d9a8b7ea343cf83c7 P 16e21272f643412cada52b5893cae14a [CANDIDATE]: Term 1 election: Requested vote from peers be50fccdbbb84336aac97309330b9050 (127.17.110.3:44211), 84f83b8cfb234aa5a779d69106186dea (127.17.110.2:45081)
I20260807 18:19:37.338850 12420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e212944892ef4b6d9a8b7ea343cf83c7" candidate_uuid: "16e21272f643412cada52b5893cae14a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be50fccdbbb84336aac97309330b9050"
I20260807 18:19:37.338980 12220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e212944892ef4b6d9a8b7ea343cf83c7" candidate_uuid: "16e21272f643412cada52b5893cae14a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84f83b8cfb234aa5a779d69106186dea"
I20260807 18:19:37.339125 12420 raft_consensus.cc:3037] T e212944892ef4b6d9a8b7ea343cf83c7 P be50fccdbbb84336aac97309330b9050 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:19:37.339289 12220 raft_consensus.cc:3037] T e212944892ef4b6d9a8b7ea343cf83c7 P 84f83b8cfb234aa5a779d69106186dea [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:19:37.341639 12420 raft_consensus.cc:2454] T e212944892ef4b6d9a8b7ea343cf83c7 P be50fccdbbb84336aac97309330b9050 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16e21272f643412cada52b5893cae14a in term 1.
I20260807 18:19:37.341868 12220 raft_consensus.cc:2454] T e212944892ef4b6d9a8b7ea343cf83c7 P 84f83b8cfb234aa5a779d69106186dea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16e21272f643412cada52b5893cae14a in term 1.
I20260807 18:19:37.342432 11918 leader_election.cc:302] T e212944892ef4b6d9a8b7ea343cf83c7 P 16e21272f643412cada52b5893cae14a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16e21272f643412cada52b5893cae14a, be50fccdbbb84336aac97309330b9050; no voters:
I20260807 18:19:37.342926 12532 raft_consensus.cc:2781] T e212944892ef4b6d9a8b7ea343cf83c7 P 16e21272f643412cada52b5893cae14a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:19:37.343437 12532 raft_consensus.cc:686] T e212944892ef4b6d9a8b7ea343cf83c7 P 16e21272f643412cada52b5893cae14a [term 1 LEADER]: Becoming Leader. State: Replica: 16e21272f643412cada52b5893cae14a, State: Running, Role: LEADER
I20260807 18:19:37.344259 12532 consensus_queue.cc:237] T e212944892ef4b6d9a8b7ea343cf83c7 P 16e21272f643412cada52b5893cae14a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be50fccdbbb84336aac97309330b9050" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44211 } } peers { permanent_uuid: "84f83b8cfb234aa5a779d69106186dea" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 45081 } } peers { permanent_uuid: "16e21272f643412cada52b5893cae14a" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 45755 } }
I20260807 18:19:37.350313 11795 catalog_manager.cc:5495] T e212944892ef4b6d9a8b7ea343cf83c7 P 16e21272f643412cada52b5893cae14a reported cstate change: term changed from 0 to 1, leader changed from <none> to 16e21272f643412cada52b5893cae14a (127.17.110.1). New cstate: current_term: 1 leader_uuid: "16e21272f643412cada52b5893cae14a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be50fccdbbb84336aac97309330b9050" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44211 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84f83b8cfb234aa5a779d69106186dea" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 45081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16e21272f643412cada52b5893cae14a" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 45755 } health_report { overall_health: HEALTHY } } }
I20260807 18:19:37.408380 12519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:37.401952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34394 (local address 127.17.110.1:45755)
0807 18:19:37.402214 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:37.402220 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:37.402239 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:19:37.402336 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:37.402340 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:37.402421 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:37.402677 (+ 256us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:19:37.403382 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:37.404208 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:37.405877 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:37.406313 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:37.406420 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:37.407019 (+ 599us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:19:37.407621 (+ 602us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:19:37.407772 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:37.407944 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 18:19:37.408022 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260807 18:19:37.417696 12420 raft_consensus.cc:1261] T e212944892ef4b6d9a8b7ea343cf83c7 P be50fccdbbb84336aac97309330b9050 [term 1 FOLLOWER]: Refusing update from remote peer 16e21272f643412cada52b5893cae14a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:19:37.417697 12220 raft_consensus.cc:1261] T e212944892ef4b6d9a8b7ea343cf83c7 P 84f83b8cfb234aa5a779d69106186dea [term 1 FOLLOWER]: Refusing update from remote peer 16e21272f643412cada52b5893cae14a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:19:37.418896 12535 consensus_queue.cc:1035] T e212944892ef4b6d9a8b7ea343cf83c7 P 16e21272f643412cada52b5893cae14a [LEADER]: Connected to new peer: Peer: permanent_uuid: "be50fccdbbb84336aac97309330b9050" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 44211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:19:37.419377 12532 consensus_queue.cc:1035] T e212944892ef4b6d9a8b7ea343cf83c7 P 16e21272f643412cada52b5893cae14a [LEADER]: Connected to new peer: Peer: permanent_uuid: "84f83b8cfb234aa5a779d69106186dea" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 45081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:19:37.434599 12540 mvcc.cc:205] Tried to move back new op lower bound from 7315975280291524608 to 7315975280006287360. Current Snapshot: MvccSnapshot[applied={T|T < 7315975280291524608}]
I20260807 18:19:37.442441 12541 mvcc.cc:205] Tried to move back new op lower bound from 7315975280291524608 to 7315975280006287360. Current Snapshot: MvccSnapshot[applied={T|T < 7315975280291524608}]
I20260807 18:19:37.454391 11795 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:19:37.454789 11795 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:19:37.459371 11795 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9e47390e368a45fa85a204d30b48e978: Sending DeleteTablet for 3 replicas of tablet e212944892ef4b6d9a8b7ea343cf83c7
I20260807 18:19:37.460937 12392 tablet_service.cc:1514] Processing DeleteTablet for tablet e212944892ef4b6d9a8b7ea343cf83c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:19:37 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:42026
I20260807 18:19:37.461119 11982 tablet_service.cc:1514] Processing DeleteTablet for tablet e212944892ef4b6d9a8b7ea343cf83c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:19:37 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:34390
I20260807 18:19:37.461161 12187 tablet_service.cc:1514] Processing DeleteTablet for tablet e212944892ef4b6d9a8b7ea343cf83c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:19:37 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:55816
I20260807 18:19:37.462651 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 11894
I20260807 18:19:37.475461 12555 tablet_replica.cc:330] T e212944892ef4b6d9a8b7ea343cf83c7 P be50fccdbbb84336aac97309330b9050: stopping tablet replica
I20260807 18:19:37.476300 12555 raft_consensus.cc:2229] T e212944892ef4b6d9a8b7ea343cf83c7 P be50fccdbbb84336aac97309330b9050 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:19:37.477083 12555 raft_consensus.cc:2258] T e212944892ef4b6d9a8b7ea343cf83c7 P be50fccdbbb84336aac97309330b9050 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:19:37.478197 12557 tablet_replica.cc:330] T e212944892ef4b6d9a8b7ea343cf83c7 P 84f83b8cfb234aa5a779d69106186dea: stopping tablet replica
I20260807 18:19:37.478824 12557 raft_consensus.cc:2229] T e212944892ef4b6d9a8b7ea343cf83c7 P 84f83b8cfb234aa5a779d69106186dea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:19:37.479446 12557 raft_consensus.cc:2258] T e212944892ef4b6d9a8b7ea343cf83c7 P 84f83b8cfb234aa5a779d69106186dea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:19:37.480046 12555 ts_tablet_manager.cc:1905] T e212944892ef4b6d9a8b7ea343cf83c7 P be50fccdbbb84336aac97309330b9050: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:19:37.482439 12557 ts_tablet_manager.cc:1905] T e212944892ef4b6d9a8b7ea343cf83c7 P 84f83b8cfb234aa5a779d69106186dea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:19:37.483016 12556 tablet_replica.cc:330] T e212944892ef4b6d9a8b7ea343cf83c7 P 16e21272f643412cada52b5893cae14a: stopping tablet replica
I20260807 18:19:37.483521 12556 raft_consensus.cc:2229] T e212944892ef4b6d9a8b7ea343cf83c7 P 16e21272f643412cada52b5893cae14a [term 1 LEADER]: Raft consensus shutting down.
Aug 07 18:19:37 dist-test-slave-39l8 krb5kdc[11739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.1@KRBTEST.COM
I20260807 18:19:37.487708 12555 ts_tablet_manager.cc:1918] T e212944892ef4b6d9a8b7ea343cf83c7 P be50fccdbbb84336aac97309330b9050: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:19:37.487931 12555 log.cc:1196] T e212944892ef4b6d9a8b7ea343cf83c7 P be50fccdbbb84336aac97309330b9050: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-2/wal/wals/e212944892ef4b6d9a8b7ea343cf83c7
I20260807 18:19:37.488543 12555 ts_tablet_manager.cc:1939] T e212944892ef4b6d9a8b7ea343cf83c7 P be50fccdbbb84336aac97309330b9050: Deleting consensus metadata
I20260807 18:19:37.488482 12556 raft_consensus.cc:2258] T e212944892ef4b6d9a8b7ea343cf83c7 P 16e21272f643412cada52b5893cae14a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:19:37.490825 11775 catalog_manager.cc:4841] TS be50fccdbbb84336aac97309330b9050 (127.17.110.3:44211): tablet e212944892ef4b6d9a8b7ea343cf83c7 (table test-table [id=9cd5d3c34d17468d9f88fdb890ab880c]) successfully deleted
I20260807 18:19:37.491575 12556 ts_tablet_manager.cc:1905] T e212944892ef4b6d9a8b7ea343cf83c7 P 16e21272f643412cada52b5893cae14a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:19:37.493927 12557 ts_tablet_manager.cc:1918] T e212944892ef4b6d9a8b7ea343cf83c7 P 84f83b8cfb234aa5a779d69106186dea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:19:37.494227 12557 log.cc:1196] T e212944892ef4b6d9a8b7ea343cf83c7 P 84f83b8cfb234aa5a779d69106186dea: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126429568977-17848-0/minicluster-data/ts-1/wal/wals/e212944892ef4b6d9a8b7ea343cf83c7
I20260807 18:19:37.494439 12519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:37.463986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34400 (local address 127.17.110.1:45755)
0807 18:19:37.464175 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:37.464182 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:37.469454 (+ 5272us) server_negotiation.cc:407] Connection header received
0807 18:19:37.469714 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:37.469720 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:37.469835 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:37.469966 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:37.472941 (+ 2975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:37.474320 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:37.477812 (+ 3492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:37.478646 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:37.478861 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:37.485290 (+ 6429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:37.485309 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:37.485328 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:19:37.485384 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:37.490147 (+ 4763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:37.491364 (+ 1217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:37.491375 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:37.491379 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:37.491579 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:37.492145 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:37.492150 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:37.492153 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:37.492680 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:37.493281 (+ 601us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:37.493666 (+ 385us) server_negotiation.cc:299] Negotiation successful
0807 18:19:37.493987 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:19:37.495324 12557 ts_tablet_manager.cc:1939] T e212944892ef4b6d9a8b7ea343cf83c7 P 84f83b8cfb234aa5a779d69106186dea: Deleting consensus metadata
I20260807 18:19:37.496253 12077 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:19:37.497742 11775 catalog_manager.cc:4841] TS 84f83b8cfb234aa5a779d69106186dea (127.17.110.2:45081): tablet e212944892ef4b6d9a8b7ea343cf83c7 (table test-table [id=9cd5d3c34d17468d9f88fdb890ab880c]) successfully deleted
I20260807 18:19:37.601254 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 11894
I20260807 18:19:37.627271 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 12095
W20260807 18:19:37.627617 11775 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.110.1:45755 (error 108)
I20260807 18:19:37.630020 12562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:37.629045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:45755 (local address 127.0.0.1:34402)
0807 18:19:37.629571 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:37.629743 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:45755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":252,"threads_started":1}
W20260807 18:19:37.630378 11775 catalog_manager.cc:4584] TS 16e21272f643412cada52b5893cae14a (127.17.110.1:45755): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e212944892ef4b6d9a8b7ea343cf83c7: Network error: Client connection negotiation failed: client connection to 127.17.110.1:45755: connect: Connection refused (error 111)
Aug 07 18:19:37 dist-test-slave-39l8 krb5kdc[11739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.2@KRBTEST.COM
I20260807 18:19:37.645867 12513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:37.628240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55832 (local address 127.17.110.2:45081)
0807 18:19:37.628445 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:37.628451 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:37.628476 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:19:37.628554 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:37.628559 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:37.628638 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:37.628795 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:37.630613 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:37.631540 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:37.633145 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:37.633611 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:37.633773 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:37.638950 (+ 5177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:37.638968 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:37.638986 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:19:37.639034 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:37.642181 (+ 3147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:37.643294 (+ 1113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:37.643303 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:37.643306 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:37.643477 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:37.643947 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:37.643953 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:37.643959 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:37.644426 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:37.644792 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:37.645176 (+ 384us) server_negotiation.cc:299] Negotiation successful
0807 18:19:37.645451 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 18:19:37.647081 12282 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:19:37.684242 12562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:37.683601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:45755 (local address 127.0.0.1:34408)
0807 18:19:37.683899 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:37.684020 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:45755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260807 18:19:37.685544 12562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:37.685073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:45755 (local address 127.0.0.1:34424)
0807 18:19:37.685257 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:37.685361 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:45755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 18:19:37.740494 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 12095
I20260807 18:19:37.746896 12562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:37.746283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:45755 (local address 127.0.0.1:34430)
0807 18:19:37.746547 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:37.746661 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:45755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260807 18:19:37.748615 12562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:37.747995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:45755 (local address 127.0.0.1:34444)
0807 18:19:37.748242 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:37.748370 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:45755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 18:19:37.767730 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 12300
Aug 07 18:19:37 dist-test-slave-39l8 krb5kdc[11739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.3@KRBTEST.COM
I20260807 18:19:37.787017 12507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:37.768698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42036 (local address 127.17.110.3:44211)
0807 18:19:37.768892 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:37.768897 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:37.768918 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:19:37.769026 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:37.769030 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:37.769103 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:37.769219 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:37.770946 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:37.771748 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:37.773295 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:37.773779 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:37.773895 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:37.780038 (+ 6143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:37.780057 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:37.780076 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:19:37.780167 (+ 91us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:37.783647 (+ 3480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:37.784649 (+ 1002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:37.784659 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:37.784661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:37.784832 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:37.785292 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:37.785296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:37.785298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:37.785731 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:37.785979 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:37.786398 (+ 419us) server_negotiation.cc:299] Negotiation successful
0807 18:19:37.786594 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 18:19:37.787789 12482 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:19:37.829298 12562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:37.828709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:45755 (local address 127.0.0.1:34452)
0807 18:19:37.828987 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:37.829091 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:45755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260807 18:19:37.830598 12562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:37.830102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:45755 (local address 127.0.0.1:34460)
0807 18:19:37.830328 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:37.830430 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:45755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260807 18:19:37.882010 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 12300
I20260807 18:19:37.911074 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 11752
I20260807 18:19:37.912424 11847 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:19:38.030256 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 11752
2026-08-07T18:19:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (5399 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T18:19:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:19:38Z Disabled control of system clock
I20260807 18:19:38.140269 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:43955
--webserver_interface=127.17.110.62
--webserver_port=0
--builtin_ntp_servers=127.17.110.20:46853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:43955
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:19:38.510129 12571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:19:38.510466 12571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:19:38.510608 12571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:19:38.518307 12571 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:19:38.518440 12571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:19:38.518479 12571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:19:38.518532 12571 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:19:38.518577 12571 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:19:38.528564 12571 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:46853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:43955
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:43955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:19:38.529906 12571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:19:38.531818 12571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:19:38.545401 12581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:38.545584 12580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:38.545666 12583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:19:38.546833 12571 server_base.cc:1034] running on GCE node
I20260807 18:19:38.547828 12571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:19:38.549422 12571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:19:38.550750 12571 hybrid_clock.cc:648] HybridClock initialized: now 1786126778550639 us; error 58 us; skew 500 ppm
I20260807 18:19:38.551159 12571 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:19:38.553927 12571 webserver.cc:460] Webserver started at http://127.17.110.62:37709/ using document root <none> and password file <none>
I20260807 18:19:38.554778 12571 fs_manager.cc:359] Metadata directory not provided
I20260807 18:19:38.554926 12571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:19:38.555292 12571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:19:38.558462 12571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "d0c9af76d21f4cb4a47f1e3538983e71"
format_stamp: "Formatted at 2026-08-07 18:19:38 on dist-test-slave-39l8"
server_key: "a94e7527cc0209d0a9e3c2b3df425a0a"
server_key_iv: "f450eb1d6c4fa209f5b16d7935f88715"
server_key_version: "encryptionkey@0"
I20260807 18:19:38.559444 12571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "d0c9af76d21f4cb4a47f1e3538983e71"
format_stamp: "Formatted at 2026-08-07 18:19:38 on dist-test-slave-39l8"
server_key: "a94e7527cc0209d0a9e3c2b3df425a0a"
server_key_iv: "f450eb1d6c4fa209f5b16d7935f88715"
server_key_version: "encryptionkey@0"
I20260807 18:19:38.565402 12571 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 18:19:38.569444 12594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:38.571030 12571 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:19:38.571264 12571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "d0c9af76d21f4cb4a47f1e3538983e71"
format_stamp: "Formatted at 2026-08-07 18:19:38 on dist-test-slave-39l8"
server_key: "a94e7527cc0209d0a9e3c2b3df425a0a"
server_key_iv: "f450eb1d6c4fa209f5b16d7935f88715"
server_key_version: "encryptionkey@0"
I20260807 18:19:38.571524 12571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:19:38.590574 12571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:19:38.591559 12571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:19:38.591893 12571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:19:38.613476 12571 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:43955
I20260807 18:19:38.613517 12668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:43955 every 8 connection(s)
I20260807 18:19:38.615170 12571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:19:38.619493 12670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:19:38.625559 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 12571
I20260807 18:19:38.625799 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:19:38.626312 17848 external_mini_cluster.cc:1428] Setting key 83645f0de62823fa83c9e899f5687020
I20260807 18:19:38.632025 12670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0c9af76d21f4cb4a47f1e3538983e71: Bootstrap starting.
I20260807 18:19:38.636128 12670 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0c9af76d21f4cb4a47f1e3538983e71: Neither blocks nor log segments found. Creating new log.
I20260807 18:19:38.637591 12670 log.cc:824] T 00000000000000000000000000000000 P d0c9af76d21f4cb4a47f1e3538983e71: Log is configured to *not* fsync() on all Append() calls
I20260807 18:19:38.641505 12670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0c9af76d21f4cb4a47f1e3538983e71: No bootstrap required, opened a new log
I20260807 18:19:38.642992 12674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:38.628723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53754 (local address 127.17.110.62:43955)
0807 18:19:38.629982 (+ 1259us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:38.629997 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:38.630047 (+ 50us) server_negotiation.cc:407] Connection header received
0807 18:19:38.631908 (+ 1861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:38.631935 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:38.632508 (+ 573us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:38.632915 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:38.633637 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:38.635139 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:38.637341 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:38.638363 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:38.638613 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:38.639864 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:38.639875 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:38.639898 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:19:38.640711 (+ 813us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:38.640834 (+ 123us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:19:38.640863 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:38.640995 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:38.641227 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 18:19:38.641582 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":778,"thread_start_us":504,"threads_started":1}
I20260807 18:19:38.650617 12670 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0c9af76d21f4cb4a47f1e3538983e71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c9af76d21f4cb4a47f1e3538983e71" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 43955 } }
I20260807 18:19:38.651137 12670 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0c9af76d21f4cb4a47f1e3538983e71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:19:38.651232 12670 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0c9af76d21f4cb4a47f1e3538983e71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0c9af76d21f4cb4a47f1e3538983e71, State: Initialized, Role: FOLLOWER
I20260807 18:19:38.651822 12670 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0c9af76d21f4cb4a47f1e3538983e71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c9af76d21f4cb4a47f1e3538983e71" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 43955 } }
I20260807 18:19:38.652071 12670 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0c9af76d21f4cb4a47f1e3538983e71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:19:38.652248 12670 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0c9af76d21f4cb4a47f1e3538983e71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:19:38.652464 12670 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0c9af76d21f4cb4a47f1e3538983e71 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:19:38.655170 12670 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0c9af76d21f4cb4a47f1e3538983e71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c9af76d21f4cb4a47f1e3538983e71" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 43955 } }
I20260807 18:19:38.655725 12670 leader_election.cc:302] T 00000000000000000000000000000000 P d0c9af76d21f4cb4a47f1e3538983e71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0c9af76d21f4cb4a47f1e3538983e71; no voters:
I20260807 18:19:38.656374 12670 leader_election.cc:288] T 00000000000000000000000000000000 P d0c9af76d21f4cb4a47f1e3538983e71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:19:38.656625 12678 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0c9af76d21f4cb4a47f1e3538983e71 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:19:38.657481 12678 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0c9af76d21f4cb4a47f1e3538983e71 [term 1 LEADER]: Becoming Leader. State: Replica: d0c9af76d21f4cb4a47f1e3538983e71, State: Running, Role: LEADER
I20260807 18:19:38.658160 12678 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0c9af76d21f4cb4a47f1e3538983e71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c9af76d21f4cb4a47f1e3538983e71" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 43955 } }
I20260807 18:19:38.658878 12670 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0c9af76d21f4cb4a47f1e3538983e71 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:19:38.661715 12679 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0c9af76d21f4cb4a47f1e3538983e71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0c9af76d21f4cb4a47f1e3538983e71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c9af76d21f4cb4a47f1e3538983e71" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 43955 } } }
I20260807 18:19:38.662097 12679 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0c9af76d21f4cb4a47f1e3538983e71 [sys.catalog]: This master's current role is: LEADER
I20260807 18:19:38.663254 12691 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:19:38.663045 12680 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0c9af76d21f4cb4a47f1e3538983e71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0c9af76d21f4cb4a47f1e3538983e71. Latest consensus state: current_term: 1 leader_uuid: "d0c9af76d21f4cb4a47f1e3538983e71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c9af76d21f4cb4a47f1e3538983e71" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 43955 } } }
I20260807 18:19:38.664610 12680 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0c9af76d21f4cb4a47f1e3538983e71 [sys.catalog]: This master's current role is: LEADER
I20260807 18:19:38.669735 12691 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:19:38.680274 12691 catalog_manager.cc:1347] Generated new cluster ID: eb007108492a420c905496ae21ae9a99
I20260807 18:19:38.680419 12691 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:19:38.701838 12691 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:19:38.703401 12691 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:19:38.713090 12691 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0c9af76d21f4cb4a47f1e3538983e71: Generated new TSK 0
I20260807 18:19:38.714299 12691 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:19:38.731597 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.1:0
--local_ip_for_outbound_sockets=127.17.110.1
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:43955
--builtin_ntp_servers=127.17.110.20:46853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:19:39.102325 12711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:19:39.102715 12711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:19:39.102924 12711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:19:39.110679 12711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:19:39.110911 12711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:19:39.111110 12711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.1
I20260807 18:19:39.121016 12711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:46853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:43955
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.1
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:19:39.122469 12711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:19:39.124475 12711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:19:39.139276 12719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:39.139365 12723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:39.139384 12717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:19:39.140843 12711 server_base.cc:1034] running on GCE node
I20260807 18:19:39.141708 12711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:19:39.143002 12711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:19:39.144264 12711 hybrid_clock.cc:648] HybridClock initialized: now 1786126779144197 us; error 52 us; skew 500 ppm
I20260807 18:19:39.144650 12711 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:19:39.147473 12711 webserver.cc:460] Webserver started at http://127.17.110.1:46835/ using document root <none> and password file <none>
I20260807 18:19:39.148228 12711 fs_manager.cc:359] Metadata directory not provided
I20260807 18:19:39.148329 12711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:19:39.148635 12711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:19:39.151789 12711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-0/data/instance:
uuid: "9eb0135691d246f2944aaee5d68f999b"
format_stamp: "Formatted at 2026-08-07 18:19:39 on dist-test-slave-39l8"
server_key: "151eb785ecc813bbc55866a2b14025c1"
server_key_iv: "98fc343a20c19887f60131ca71af790a"
server_key_version: "encryptionkey@0"
I20260807 18:19:39.152686 12711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance:
uuid: "9eb0135691d246f2944aaee5d68f999b"
format_stamp: "Formatted at 2026-08-07 18:19:39 on dist-test-slave-39l8"
server_key: "151eb785ecc813bbc55866a2b14025c1"
server_key_iv: "98fc343a20c19887f60131ca71af790a"
server_key_version: "encryptionkey@0"
I20260807 18:19:39.158635 12711 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260807 18:19:39.163038 12731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:39.164716 12711 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:19:39.164932 12711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-0/wal
uuid: "9eb0135691d246f2944aaee5d68f999b"
format_stamp: "Formatted at 2026-08-07 18:19:39 on dist-test-slave-39l8"
server_key: "151eb785ecc813bbc55866a2b14025c1"
server_key_iv: "98fc343a20c19887f60131ca71af790a"
server_key_version: "encryptionkey@0"
I20260807 18:19:39.165194 12711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:19:39.182379 12711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:19:39.183409 12711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:19:39.183758 12711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:19:39.184878 12711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:19:39.186707 12711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:19:39.186894 12711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 18:19:39.187052 12711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:19:39.187150 12711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:39.224452 12711 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.1:41941
I20260807 18:19:39.224563 12892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.1:41941 every 8 connection(s)
I20260807 18:19:39.225922 12711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
I20260807 18:19:39.228240 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 12711
I20260807 18:19:39.228441 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance
I20260807 18:19:39.229076 17848 external_mini_cluster.cc:1428] Setting key 3f349dafc6e23991ef724c889b6a0feb
I20260807 18:19:39.232506 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.2:0
--local_ip_for_outbound_sockets=127.17.110.2
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:43955
--builtin_ntp_servers=127.17.110.20:46853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:19:39.241371 12896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:39.229084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:58367 (local address 127.17.110.62:43955)
0807 18:19:39.229624 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:39.229629 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:39.231489 (+ 1860us) server_negotiation.cc:407] Connection header received
0807 18:19:39.233823 (+ 2334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:39.233829 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:39.233937 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:39.234074 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:39.235381 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:39.236190 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:39.237854 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:39.238299 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:39.238458 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:39.240341 (+ 1883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:39.240346 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:39.240350 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:19:39.240375 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:39.240420 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:19:39.240434 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:39.240556 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:39.240842 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 18:19:39.240966 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":304,"threads_started":1}
I20260807 18:19:39.242594 12899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:39.229807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:43955 (local address 127.17.110.1:58367)
0807 18:19:39.231308 (+ 1501us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:39.231364 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:39.233482 (+ 2118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:39.234340 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:39.234350 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:39.234448 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:19:39.235196 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:39.235214 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:39.236378 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:39.236383 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:39.237687 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:39.237702 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:39.238578 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:39.238587 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:39.238909 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:39.239861 (+ 952us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:19:39.239865 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:19:39.239913 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:19:39.240012 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:19:39.240554 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:19:39.240569 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:19:39.240581 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 18:19:39.240809 (+ 228us) client_negotiation.cc:240] Negotiation successful
0807 18:19:39.241316 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":744,"thread_start_us":453,"threads_started":1}
I20260807 18:19:39.244593 12894 heartbeater.cc:347] Connected to a master server at 127.17.110.62:43955
I20260807 18:19:39.245019 12894 heartbeater.cc:464] Registering TS with master...
I20260807 18:19:39.246145 12894 heartbeater.cc:511] Master 127.17.110.62:43955 requested a full tablet report, sending...
I20260807 18:19:39.248698 12620 ts_manager.cc:194] Registered new tserver with Master: 9eb0135691d246f2944aaee5d68f999b (127.17.110.1:41941)
I20260807 18:19:39.250391 12620 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.110.1:58367
W20260807 18:19:39.600167 12900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:19:39.600576 12900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:19:39.600747 12900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:19:39.608342 12900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:19:39.608523 12900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:19:39.608721 12900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.2
I20260807 18:19:39.622172 12900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:46853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:43955
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.2
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:19:39.623939 12900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:19:39.626189 12900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:19:39.643101 12907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:39.643193 12912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:39.643232 12908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:19:39.643723 12900 server_base.cc:1034] running on GCE node
I20260807 18:19:39.644693 12900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:19:39.645673 12900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:19:39.646903 12900 hybrid_clock.cc:648] HybridClock initialized: now 1786126779646855 us; error 42 us; skew 500 ppm
I20260807 18:19:39.647257 12900 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:19:39.649843 12900 webserver.cc:460] Webserver started at http://127.17.110.2:34195/ using document root <none> and password file <none>
I20260807 18:19:39.650619 12900 fs_manager.cc:359] Metadata directory not provided
I20260807 18:19:39.650784 12900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:19:39.651140 12900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:19:39.654210 12900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-1/data/instance:
uuid: "64fe9b685b894dee8ae689ccddf8b458"
format_stamp: "Formatted at 2026-08-07 18:19:39 on dist-test-slave-39l8"
server_key: "90743eb670438cd4de4f0afc588df151"
server_key_iv: "6e1cf49cc067a288332e3ec1b8c9ea06"
server_key_version: "encryptionkey@0"
I20260807 18:19:39.655184 12900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance:
uuid: "64fe9b685b894dee8ae689ccddf8b458"
format_stamp: "Formatted at 2026-08-07 18:19:39 on dist-test-slave-39l8"
server_key: "90743eb670438cd4de4f0afc588df151"
server_key_iv: "6e1cf49cc067a288332e3ec1b8c9ea06"
server_key_version: "encryptionkey@0"
I20260807 18:19:39.661049 12900 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:19:39.665066 12921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:39.666747 12900 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:19:39.666951 12900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-1/wal
uuid: "64fe9b685b894dee8ae689ccddf8b458"
format_stamp: "Formatted at 2026-08-07 18:19:39 on dist-test-slave-39l8"
server_key: "90743eb670438cd4de4f0afc588df151"
server_key_iv: "6e1cf49cc067a288332e3ec1b8c9ea06"
server_key_version: "encryptionkey@0"
I20260807 18:19:39.667253 12900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:19:39.691299 12900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:19:39.692246 12900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:19:39.692610 12900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:19:39.693696 12900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:19:39.695386 12900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:19:39.695518 12900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:39.695662 12900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:19:39.695757 12900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:39.731693 12900 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.2:40279
I20260807 18:19:39.731792 13085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.2:40279 every 8 connection(s)
I20260807 18:19:39.733146 12900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
I20260807 18:19:39.739948 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 12900
I20260807 18:19:39.740193 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance
I20260807 18:19:39.740710 17848 external_mini_cluster.cc:1428] Setting key ba5e149c5a69a6fef46520d672a7db7b
I20260807 18:19:39.743957 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.3:0
--local_ip_for_outbound_sockets=127.17.110.3
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:43955
--builtin_ntp_servers=127.17.110.20:46853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:19:39.747774 12896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:39.735942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:55543 (local address 127.17.110.62:43955)
0807 18:19:39.736170 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:39.736177 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:39.737720 (+ 1543us) server_negotiation.cc:407] Connection header received
0807 18:19:39.739692 (+ 1972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:39.739698 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:39.739812 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:39.739939 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:39.741349 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:39.742241 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:39.744596 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:39.745012 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:39.745123 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:39.746986 (+ 1863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:39.746991 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:39.746994 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:19:39.747016 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:39.747048 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:19:39.747060 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:39.747131 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:39.747418 (+ 287us) server_negotiation.cc:299] Negotiation successful
0807 18:19:39.747521 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 18:19:39.749361 13091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:39.736557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:43955 (local address 127.17.110.2:55543)
0807 18:19:39.737518 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:39.737586 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:39.739376 (+ 1790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:39.740229 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:39.740238 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:39.740383 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:19:39.741173 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:39.741199 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:39.742409 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:39.742415 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:39.744428 (+ 2013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:39.744447 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:39.745271 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:39.745281 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:39.745680 (+ 399us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:39.746521 (+ 841us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:19:39.746527 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:19:39.746596 (+ 69us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:19:39.746752 (+ 156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:19:39.747161 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:19:39.747175 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:19:39.747187 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 18:19:39.747381 (+ 194us) client_negotiation.cc:240] Negotiation successful
0807 18:19:39.747875 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":313,"threads_started":1}
I20260807 18:19:39.750993 13088 heartbeater.cc:347] Connected to a master server at 127.17.110.62:43955
I20260807 18:19:39.751343 13088 heartbeater.cc:464] Registering TS with master...
I20260807 18:19:39.752326 13088 heartbeater.cc:511] Master 127.17.110.62:43955 requested a full tablet report, sending...
I20260807 18:19:39.754065 12620 ts_manager.cc:194] Registered new tserver with Master: 64fe9b685b894dee8ae689ccddf8b458 (127.17.110.2:40279)
I20260807 18:19:39.754969 12620 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.110.2:55543
W20260807 18:19:40.113456 13092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:19:40.113795 13092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:19:40.113936 13092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:19:40.121544 13092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:19:40.121708 13092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:19:40.121902 13092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.3
I20260807 18:19:40.131544 13092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:46853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:43955
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.3
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:19:40.133020 13092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:19:40.135084 13092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:19:40.148108 13098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:40.149068 13099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:40.150409 13102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:19:40.151046 13092 server_base.cc:1034] running on GCE node
I20260807 18:19:40.151777 13092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:19:40.152913 13092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:19:40.154188 13092 hybrid_clock.cc:648] HybridClock initialized: now 1786126780154141 us; error 47 us; skew 500 ppm
I20260807 18:19:40.154577 13092 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:19:40.157353 13092 webserver.cc:460] Webserver started at http://127.17.110.3:44013/ using document root <none> and password file <none>
I20260807 18:19:40.158193 13092 fs_manager.cc:359] Metadata directory not provided
I20260807 18:19:40.158355 13092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:19:40.158788 13092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:19:40.162298 13092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-2/data/instance:
uuid: "fec595231b604f258412e1685da505d0"
format_stamp: "Formatted at 2026-08-07 18:19:40 on dist-test-slave-39l8"
server_key: "9021ade5dc252027983ca16d15c0f870"
server_key_iv: "936443e598d640b1a96a27bb9b84c91b"
server_key_version: "encryptionkey@0"
I20260807 18:19:40.163275 13092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance:
uuid: "fec595231b604f258412e1685da505d0"
format_stamp: "Formatted at 2026-08-07 18:19:40 on dist-test-slave-39l8"
server_key: "9021ade5dc252027983ca16d15c0f870"
server_key_iv: "936443e598d640b1a96a27bb9b84c91b"
server_key_version: "encryptionkey@0"
I20260807 18:19:40.169760 13092 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 18:19:40.174297 13113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:40.176185 13092 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:19:40.176432 13092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-2/wal
uuid: "fec595231b604f258412e1685da505d0"
format_stamp: "Formatted at 2026-08-07 18:19:40 on dist-test-slave-39l8"
server_key: "9021ade5dc252027983ca16d15c0f870"
server_key_iv: "936443e598d640b1a96a27bb9b84c91b"
server_key_version: "encryptionkey@0"
I20260807 18:19:40.176806 13092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:19:40.192930 13092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:19:40.193858 13092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:19:40.194195 13092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:19:40.195348 13092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:19:40.196975 13092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:19:40.197073 13092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:40.197235 13092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:19:40.197331 13092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:40.233513 13092 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.3:35251
I20260807 18:19:40.233561 13297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.3:35251 every 8 connection(s)
I20260807 18:19:40.235469 13092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
I20260807 18:19:40.240765 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 13092
I20260807 18:19:40.240972 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance
I20260807 18:19:40.241436 17848 external_mini_cluster.cc:1428] Setting key ba0b87cff60f0a0db2168b473fead25a
I20260807 18:19:40.249972 12896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:40.238416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:52817 (local address 127.17.110.62:43955)
0807 18:19:40.238650 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:40.238655 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:40.240248 (+ 1593us) server_negotiation.cc:407] Connection header received
0807 18:19:40.242338 (+ 2090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:40.242343 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:40.242420 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:40.242548 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:40.244070 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.245132 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.247052 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.247473 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.247579 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:40.249134 (+ 1555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:40.249139 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:40.249142 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:19:40.249185 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:40.249218 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:19:40.249231 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:40.249307 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:40.249597 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 18:19:40.249696 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 18:19:40.251303 13300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:40.239033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:43955 (local address 127.17.110.3:52817)
0807 18:19:40.240032 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:40.240086 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:40.241953 (+ 1867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:40.242908 (+ 955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:40.242921 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:40.243057 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:19:40.243878 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:40.243902 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.245316 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.245320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:40.246859 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:40.246877 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.247734 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.247740 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:40.247967 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:40.248730 (+ 763us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:19:40.248733 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:19:40.248784 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:19:40.248886 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:19:40.249358 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:19:40.249369 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:19:40.249376 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 18:19:40.249544 (+ 168us) client_negotiation.cc:240] Negotiation successful
0807 18:19:40.249987 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":308,"threads_started":1}
I20260807 18:19:40.252966 13298 heartbeater.cc:347] Connected to a master server at 127.17.110.62:43955
I20260807 18:19:40.253330 13298 heartbeater.cc:464] Registering TS with master...
I20260807 18:19:40.254176 13298 heartbeater.cc:511] Master 127.17.110.62:43955 requested a full tablet report, sending...
I20260807 18:19:40.254537 12894 heartbeater.cc:503] Master 127.17.110.62:43955 was elected leader, sending a full tablet report...
I20260807 18:19:40.256150 12619 ts_manager.cc:194] Registered new tserver with Master: fec595231b604f258412e1685da505d0 (127.17.110.3:35251)
I20260807 18:19:40.257035 12619 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.110.3:52817
I20260807 18:19:40.257913 17848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:19:40.269548 12896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:40.262679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53760 (local address 127.17.110.62:43955)
0807 18:19:40.262893 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:40.262898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:40.263288 (+ 390us) server_negotiation.cc:407] Connection header received
0807 18:19:40.263416 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:40.263420 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:40.263487 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:40.263580 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:40.264164 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.264916 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.266642 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.267271 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.267397 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:40.268644 (+ 1247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:40.268670 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:40.268674 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:19:40.268697 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:40.268733 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:19:40.268746 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:40.268871 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:40.269043 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 18:19:40.269240 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 18:19:40.274567 12620 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53760:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:19:40.283239 12620 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:19:40.323931 13319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:40.298276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59930 (local address 127.17.110.3:35251)
0807 18:19:40.311403 (+ 13127us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:40.311418 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:40.312884 (+ 1466us) server_negotiation.cc:407] Connection header received
0807 18:19:40.313005 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:40.313014 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:40.313315 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:40.314150 (+ 835us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:19:40.315453 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.317195 (+ 1742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.321121 (+ 3926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.322542 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.322758 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:40.323121 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:40.323309 (+ 188us) server_negotiation.cc:299] Negotiation successful
0807 18:19:40.323499 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12848,"thread_start_us":12654,"threads_started":1}
I20260807 18:19:40.323966 13316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:40.297810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:35251 (local address 127.0.0.1:59930)
0807 18:19:40.312235 (+ 14425us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:40.312335 (+ 100us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:40.312622 (+ 287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:40.313601 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:40.313613 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:40.313653 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:19:40.315188 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:40.315206 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.317411 (+ 2205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.317416 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:40.320774 (+ 3358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:40.320795 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.322797 (+ 2002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.322804 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:40.323026 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:40.323095 (+ 69us) client_negotiation.cc:769] Sending connection context
0807 18:19:40.323241 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 18:19:40.323479 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12776,"mutex_wait_us":175,"thread_start_us":14402,"threads_started":1}
I20260807 18:19:40.324785 13315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:40.298340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:40279 (local address 127.0.0.1:43974)
0807 18:19:40.312550 (+ 14210us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:40.312571 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:40.312708 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:40.313890 (+ 1182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:40.313895 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:40.313928 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:19:40.314612 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:40.314631 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.317408 (+ 2777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.317414 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:40.321118 (+ 3704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:40.321148 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.323784 (+ 2636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.323794 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:40.324022 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:40.324063 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 18:19:40.324163 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 18:19:40.324310 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14068,"mutex_wait_us":521,"thread_start_us":12072,"threads_started":1}
I20260807 18:19:40.325655 13323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:40.299544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41856 (local address 127.17.110.1:41941)
0807 18:19:40.314615 (+ 15071us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:40.314629 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:40.315563 (+ 934us) server_negotiation.cc:407] Connection header received
0807 18:19:40.315846 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:40.315854 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:40.316169 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:40.316396 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:19:40.317478 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.318699 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.321658 (+ 2959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.323322 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.323474 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:40.323770 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:40.325014 (+ 1244us) server_negotiation.cc:299] Negotiation successful
0807 18:19:40.325236 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14833,"thread_start_us":14674,"threads_started":1}
I20260807 18:19:40.325739 13322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:40.299146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43974 (local address 127.17.110.2:40279)
0807 18:19:40.311976 (+ 12830us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:40.311985 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:40.312883 (+ 898us) server_negotiation.cc:407] Connection header received
0807 18:19:40.313004 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:40.313014 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:40.313312 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:40.313538 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:19:40.315227 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.317186 (+ 1959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.321458 (+ 4272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.323171 (+ 1713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.323307 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:40.323570 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:40.325145 (+ 1575us) server_negotiation.cc:299] Negotiation successful
0807 18:19:40.325327 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12654,"thread_start_us":12517,"threads_started":1}
I20260807 18:19:40.326912 13321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:40.298752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:41941 (local address 127.0.0.1:41856)
0807 18:19:40.315402 (+ 16650us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:40.315426 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:40.315595 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:40.316716 (+ 1121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:40.316720 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:40.316753 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:19:40.317254 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:40.317271 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.318936 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.318943 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:40.321374 (+ 2431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:40.321393 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.323915 (+ 2522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.323922 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:40.324348 (+ 426us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:40.324382 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 18:19:40.324472 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 18:19:40.324617 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16483,"mutex_wait_us":274,"thread_start_us":16252,"threads_started":1}
I20260807 18:19:40.328891 13191 tablet_service.cc:1467] Processing CreateTablet for tablet 51b0cbfd37c44076afa99555f0e25f62 (DEFAULT_TABLE table=test-table [id=e04002a694c74e06af9f399d635569e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:19:40.329355 12995 tablet_service.cc:1467] Processing CreateTablet for tablet 51b0cbfd37c44076afa99555f0e25f62 (DEFAULT_TABLE table=test-table [id=e04002a694c74e06af9f399d635569e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:19:40.330646 13191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51b0cbfd37c44076afa99555f0e25f62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:19:40.330960 12995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51b0cbfd37c44076afa99555f0e25f62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:19:40.332607 12795 tablet_service.cc:1467] Processing CreateTablet for tablet 51b0cbfd37c44076afa99555f0e25f62 (DEFAULT_TABLE table=test-table [id=e04002a694c74e06af9f399d635569e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:19:40.334358 12795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51b0cbfd37c44076afa99555f0e25f62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:19:40.344676 13325 tablet_bootstrap.cc:492] T 51b0cbfd37c44076afa99555f0e25f62 P fec595231b604f258412e1685da505d0: Bootstrap starting.
I20260807 18:19:40.349298 13325 tablet_bootstrap.cc:654] T 51b0cbfd37c44076afa99555f0e25f62 P fec595231b604f258412e1685da505d0: Neither blocks nor log segments found. Creating new log.
I20260807 18:19:40.350522 13325 log.cc:824] T 51b0cbfd37c44076afa99555f0e25f62 P fec595231b604f258412e1685da505d0: Log is configured to *not* fsync() on all Append() calls
I20260807 18:19:40.353806 13329 tablet_bootstrap.cc:492] T 51b0cbfd37c44076afa99555f0e25f62 P 9eb0135691d246f2944aaee5d68f999b: Bootstrap starting.
I20260807 18:19:40.354099 13325 tablet_bootstrap.cc:492] T 51b0cbfd37c44076afa99555f0e25f62 P fec595231b604f258412e1685da505d0: No bootstrap required, opened a new log
I20260807 18:19:40.354418 13325 ts_tablet_manager.cc:1397] T 51b0cbfd37c44076afa99555f0e25f62 P fec595231b604f258412e1685da505d0: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.005s
I20260807 18:19:40.354455 13326 tablet_bootstrap.cc:492] T 51b0cbfd37c44076afa99555f0e25f62 P 64fe9b685b894dee8ae689ccddf8b458: Bootstrap starting.
I20260807 18:19:40.358218 13326 tablet_bootstrap.cc:654] T 51b0cbfd37c44076afa99555f0e25f62 P 64fe9b685b894dee8ae689ccddf8b458: Neither blocks nor log segments found. Creating new log.
I20260807 18:19:40.358258 13329 tablet_bootstrap.cc:654] T 51b0cbfd37c44076afa99555f0e25f62 P 9eb0135691d246f2944aaee5d68f999b: Neither blocks nor log segments found. Creating new log.
I20260807 18:19:40.359766 13329 log.cc:824] T 51b0cbfd37c44076afa99555f0e25f62 P 9eb0135691d246f2944aaee5d68f999b: Log is configured to *not* fsync() on all Append() calls
I20260807 18:19:40.359781 13326 log.cc:824] T 51b0cbfd37c44076afa99555f0e25f62 P 64fe9b685b894dee8ae689ccddf8b458: Log is configured to *not* fsync() on all Append() calls
I20260807 18:19:40.363917 13326 tablet_bootstrap.cc:492] T 51b0cbfd37c44076afa99555f0e25f62 P 64fe9b685b894dee8ae689ccddf8b458: No bootstrap required, opened a new log
I20260807 18:19:40.364013 13329 tablet_bootstrap.cc:492] T 51b0cbfd37c44076afa99555f0e25f62 P 9eb0135691d246f2944aaee5d68f999b: No bootstrap required, opened a new log
I20260807 18:19:40.364281 13326 ts_tablet_manager.cc:1397] T 51b0cbfd37c44076afa99555f0e25f62 P 64fe9b685b894dee8ae689ccddf8b458: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260807 18:19:40.364349 13329 ts_tablet_manager.cc:1397] T 51b0cbfd37c44076afa99555f0e25f62 P 9eb0135691d246f2944aaee5d68f999b: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.005s
I20260807 18:19:40.366788 13325 raft_consensus.cc:348] T 51b0cbfd37c44076afa99555f0e25f62 P fec595231b604f258412e1685da505d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec595231b604f258412e1685da505d0" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 35251 } } peers { permanent_uuid: "64fe9b685b894dee8ae689ccddf8b458" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40279 } } peers { permanent_uuid: "9eb0135691d246f2944aaee5d68f999b" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 41941 } }
I20260807 18:19:40.367357 13325 raft_consensus.cc:374] T 51b0cbfd37c44076afa99555f0e25f62 P fec595231b604f258412e1685da505d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:19:40.367467 13325 raft_consensus.cc:729] T 51b0cbfd37c44076afa99555f0e25f62 P fec595231b604f258412e1685da505d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fec595231b604f258412e1685da505d0, State: Initialized, Role: FOLLOWER
I20260807 18:19:40.368383 13325 consensus_queue.cc:260] T 51b0cbfd37c44076afa99555f0e25f62 P fec595231b604f258412e1685da505d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec595231b604f258412e1685da505d0" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 35251 } } peers { permanent_uuid: "64fe9b685b894dee8ae689ccddf8b458" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40279 } } peers { permanent_uuid: "9eb0135691d246f2944aaee5d68f999b" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 41941 } }
I20260807 18:19:40.370021 13298 heartbeater.cc:503] Master 127.17.110.62:43955 was elected leader, sending a full tablet report...
I20260807 18:19:40.370692 13325 ts_tablet_manager.cc:1428] T 51b0cbfd37c44076afa99555f0e25f62 P fec595231b604f258412e1685da505d0: Time spent starting tablet: real 0.016s user 0.007s sys 0.007s
I20260807 18:19:40.375319 13326 raft_consensus.cc:348] T 51b0cbfd37c44076afa99555f0e25f62 P 64fe9b685b894dee8ae689ccddf8b458 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec595231b604f258412e1685da505d0" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 35251 } } peers { permanent_uuid: "64fe9b685b894dee8ae689ccddf8b458" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40279 } } peers { permanent_uuid: "9eb0135691d246f2944aaee5d68f999b" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 41941 } }
I20260807 18:19:40.375833 13326 raft_consensus.cc:374] T 51b0cbfd37c44076afa99555f0e25f62 P 64fe9b685b894dee8ae689ccddf8b458 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:19:40.375982 13326 raft_consensus.cc:729] T 51b0cbfd37c44076afa99555f0e25f62 P 64fe9b685b894dee8ae689ccddf8b458 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64fe9b685b894dee8ae689ccddf8b458, State: Initialized, Role: FOLLOWER
I20260807 18:19:40.376514 13329 raft_consensus.cc:348] T 51b0cbfd37c44076afa99555f0e25f62 P 9eb0135691d246f2944aaee5d68f999b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec595231b604f258412e1685da505d0" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 35251 } } peers { permanent_uuid: "64fe9b685b894dee8ae689ccddf8b458" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40279 } } peers { permanent_uuid: "9eb0135691d246f2944aaee5d68f999b" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 41941 } }
I20260807 18:19:40.376740 13326 consensus_queue.cc:260] T 51b0cbfd37c44076afa99555f0e25f62 P 64fe9b685b894dee8ae689ccddf8b458 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec595231b604f258412e1685da505d0" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 35251 } } peers { permanent_uuid: "64fe9b685b894dee8ae689ccddf8b458" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40279 } } peers { permanent_uuid: "9eb0135691d246f2944aaee5d68f999b" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 41941 } }
I20260807 18:19:40.376991 13329 raft_consensus.cc:374] T 51b0cbfd37c44076afa99555f0e25f62 P 9eb0135691d246f2944aaee5d68f999b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:19:40.377113 13329 raft_consensus.cc:729] T 51b0cbfd37c44076afa99555f0e25f62 P 9eb0135691d246f2944aaee5d68f999b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9eb0135691d246f2944aaee5d68f999b, State: Initialized, Role: FOLLOWER
I20260807 18:19:40.377790 13329 consensus_queue.cc:260] T 51b0cbfd37c44076afa99555f0e25f62 P 9eb0135691d246f2944aaee5d68f999b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec595231b604f258412e1685da505d0" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 35251 } } peers { permanent_uuid: "64fe9b685b894dee8ae689ccddf8b458" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40279 } } peers { permanent_uuid: "9eb0135691d246f2944aaee5d68f999b" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 41941 } }
I20260807 18:19:40.378152 13088 heartbeater.cc:503] Master 127.17.110.62:43955 was elected leader, sending a full tablet report...
I20260807 18:19:40.379019 13326 ts_tablet_manager.cc:1428] T 51b0cbfd37c44076afa99555f0e25f62 P 64fe9b685b894dee8ae689ccddf8b458: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260807 18:19:40.380633 13329 ts_tablet_manager.cc:1428] T 51b0cbfd37c44076afa99555f0e25f62 P 9eb0135691d246f2944aaee5d68f999b: Time spent starting tablet: real 0.016s user 0.014s sys 0.002s
I20260807 18:19:40.434317 13335 raft_consensus.cc:482] T 51b0cbfd37c44076afa99555f0e25f62 P 9eb0135691d246f2944aaee5d68f999b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:19:40.434679 13335 raft_consensus.cc:504] T 51b0cbfd37c44076afa99555f0e25f62 P 9eb0135691d246f2944aaee5d68f999b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec595231b604f258412e1685da505d0" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 35251 } } peers { permanent_uuid: "64fe9b685b894dee8ae689ccddf8b458" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40279 } } peers { permanent_uuid: "9eb0135691d246f2944aaee5d68f999b" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 41941 } }
I20260807 18:19:40.436718 13335 leader_election.cc:288] T 51b0cbfd37c44076afa99555f0e25f62 P 9eb0135691d246f2944aaee5d68f999b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fec595231b604f258412e1685da505d0 (127.17.110.3:35251), 64fe9b685b894dee8ae689ccddf8b458 (127.17.110.2:40279)
I20260807 18:19:40.444975 13322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:40.437580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:48345 (local address 127.17.110.2:40279)
0807 18:19:40.437832 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:40.437839 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:40.438641 (+ 802us) server_negotiation.cc:407] Connection header received
0807 18:19:40.438862 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:40.438867 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:40.438967 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:40.439089 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:19:40.439881 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.440702 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.442991 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.443975 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.444154 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:40.444232 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:40.444591 (+ 359us) server_negotiation.cc:299] Negotiation successful
0807 18:19:40.444752 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260807 18:19:40.445091 13337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:40.437380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:40279 (local address 127.17.110.1:48345)
0807 18:19:40.438447 (+ 1067us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:40.438465 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:40.438665 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:40.439144 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:40.439149 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:40.439180 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:19:40.439613 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:40.439637 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.440899 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.440904 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:40.442717 (+ 1813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:40.442766 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.444185 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.444192 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:40.444392 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:40.444425 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 18:19:40.444528 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 18:19:40.444692 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":948,"mutex_wait_us":58,"thread_start_us":701,"threads_started":1}
I20260807 18:19:40.445101 13319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:40.437138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:39421 (local address 127.17.110.3:35251)
0807 18:19:40.437369 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:40.437376 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:40.438262 (+ 886us) server_negotiation.cc:407] Connection header received
0807 18:19:40.438617 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:40.438624 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:40.438747 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:40.438865 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:19:40.439655 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.440506 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.442850 (+ 2344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.444076 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.444277 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:40.444364 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:40.444728 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 18:19:40.444864 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 18:19:40.445885 13336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:40.437046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:35251 (local address 127.17.110.1:39421)
0807 18:19:40.438104 (+ 1058us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:40.438131 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:40.438409 (+ 278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:40.438918 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:40.438923 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:40.438953 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:19:40.439472 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:40.439487 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.440711 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.440716 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:40.442524 (+ 1808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:40.442540 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.444375 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.444382 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:40.444539 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:40.444566 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 18:19:40.445336 (+ 770us) client_negotiation.cc:240] Negotiation successful
0807 18:19:40.445488 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":849,"mutex_wait_us":103,"thread_start_us":589,"threads_started":1}
I20260807 18:19:40.446327 13023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51b0cbfd37c44076afa99555f0e25f62" candidate_uuid: "9eb0135691d246f2944aaee5d68f999b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64fe9b685b894dee8ae689ccddf8b458" is_pre_election: true
I20260807 18:19:40.447029 13023 raft_consensus.cc:2454] T 51b0cbfd37c44076afa99555f0e25f62 P 64fe9b685b894dee8ae689ccddf8b458 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9eb0135691d246f2944aaee5d68f999b in term 0.
I20260807 18:19:40.447126 13223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51b0cbfd37c44076afa99555f0e25f62" candidate_uuid: "9eb0135691d246f2944aaee5d68f999b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fec595231b604f258412e1685da505d0" is_pre_election: true
I20260807 18:19:40.447785 13223 raft_consensus.cc:2454] T 51b0cbfd37c44076afa99555f0e25f62 P fec595231b604f258412e1685da505d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9eb0135691d246f2944aaee5d68f999b in term 0.
I20260807 18:19:40.447983 12737 leader_election.cc:302] T 51b0cbfd37c44076afa99555f0e25f62 P 9eb0135691d246f2944aaee5d68f999b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64fe9b685b894dee8ae689ccddf8b458, 9eb0135691d246f2944aaee5d68f999b; no voters:
I20260807 18:19:40.448633 13335 raft_consensus.cc:2781] T 51b0cbfd37c44076afa99555f0e25f62 P 9eb0135691d246f2944aaee5d68f999b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:19:40.448832 13335 raft_consensus.cc:482] T 51b0cbfd37c44076afa99555f0e25f62 P 9eb0135691d246f2944aaee5d68f999b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:19:40.448952 13335 raft_consensus.cc:3037] T 51b0cbfd37c44076afa99555f0e25f62 P 9eb0135691d246f2944aaee5d68f999b [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:19:40.452354 13335 raft_consensus.cc:504] T 51b0cbfd37c44076afa99555f0e25f62 P 9eb0135691d246f2944aaee5d68f999b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec595231b604f258412e1685da505d0" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 35251 } } peers { permanent_uuid: "64fe9b685b894dee8ae689ccddf8b458" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40279 } } peers { permanent_uuid: "9eb0135691d246f2944aaee5d68f999b" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 41941 } }
I20260807 18:19:40.453104 13335 leader_election.cc:288] T 51b0cbfd37c44076afa99555f0e25f62 P 9eb0135691d246f2944aaee5d68f999b [CANDIDATE]: Term 1 election: Requested vote from peers fec595231b604f258412e1685da505d0 (127.17.110.3:35251), 64fe9b685b894dee8ae689ccddf8b458 (127.17.110.2:40279)
I20260807 18:19:40.453820 13223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51b0cbfd37c44076afa99555f0e25f62" candidate_uuid: "9eb0135691d246f2944aaee5d68f999b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fec595231b604f258412e1685da505d0"
I20260807 18:19:40.453820 13023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51b0cbfd37c44076afa99555f0e25f62" candidate_uuid: "9eb0135691d246f2944aaee5d68f999b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64fe9b685b894dee8ae689ccddf8b458"
I20260807 18:19:40.454082 13023 raft_consensus.cc:3037] T 51b0cbfd37c44076afa99555f0e25f62 P 64fe9b685b894dee8ae689ccddf8b458 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:19:40.454082 13223 raft_consensus.cc:3037] T 51b0cbfd37c44076afa99555f0e25f62 P fec595231b604f258412e1685da505d0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:19:40.457968 13223 raft_consensus.cc:2454] T 51b0cbfd37c44076afa99555f0e25f62 P fec595231b604f258412e1685da505d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9eb0135691d246f2944aaee5d68f999b in term 1.
I20260807 18:19:40.457968 13023 raft_consensus.cc:2454] T 51b0cbfd37c44076afa99555f0e25f62 P 64fe9b685b894dee8ae689ccddf8b458 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9eb0135691d246f2944aaee5d68f999b in term 1.
I20260807 18:19:40.458675 12736 leader_election.cc:302] T 51b0cbfd37c44076afa99555f0e25f62 P 9eb0135691d246f2944aaee5d68f999b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9eb0135691d246f2944aaee5d68f999b, fec595231b604f258412e1685da505d0; no voters:
I20260807 18:19:40.459208 13335 raft_consensus.cc:2781] T 51b0cbfd37c44076afa99555f0e25f62 P 9eb0135691d246f2944aaee5d68f999b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:19:40.459918 13335 raft_consensus.cc:686] T 51b0cbfd37c44076afa99555f0e25f62 P 9eb0135691d246f2944aaee5d68f999b [term 1 LEADER]: Becoming Leader. State: Replica: 9eb0135691d246f2944aaee5d68f999b, State: Running, Role: LEADER
I20260807 18:19:40.460634 13335 consensus_queue.cc:237] T 51b0cbfd37c44076afa99555f0e25f62 P 9eb0135691d246f2944aaee5d68f999b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec595231b604f258412e1685da505d0" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 35251 } } peers { permanent_uuid: "64fe9b685b894dee8ae689ccddf8b458" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40279 } } peers { permanent_uuid: "9eb0135691d246f2944aaee5d68f999b" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 41941 } }
I20260807 18:19:40.466691 12620 catalog_manager.cc:5495] T 51b0cbfd37c44076afa99555f0e25f62 P 9eb0135691d246f2944aaee5d68f999b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9eb0135691d246f2944aaee5d68f999b (127.17.110.1). New cstate: current_term: 1 leader_uuid: "9eb0135691d246f2944aaee5d68f999b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec595231b604f258412e1685da505d0" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 35251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64fe9b685b894dee8ae689ccddf8b458" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9eb0135691d246f2944aaee5d68f999b" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 41941 } health_report { overall_health: HEALTHY } } }
W20260807 18:19:40.478871 12895 tablet.cc:2369] T 51b0cbfd37c44076afa99555f0e25f62 P 9eb0135691d246f2944aaee5d68f999b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:19:40.485374 13090 tablet.cc:2369] T 51b0cbfd37c44076afa99555f0e25f62 P 64fe9b685b894dee8ae689ccddf8b458: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:19:40.487303 13299 tablet.cc:2369] T 51b0cbfd37c44076afa99555f0e25f62 P fec595231b604f258412e1685da505d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:19:40.488102 13323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:40.481967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41858 (local address 127.17.110.1:41941)
0807 18:19:40.482194 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:40.482201 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:40.482223 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:19:40.482321 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:40.482324 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:40.482433 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:40.482602 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:19:40.483030 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.483725 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.485267 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.485832 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.485939 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:40.486635 (+ 696us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:19:40.487332 (+ 697us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:19:40.487537 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:40.487782 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 18:19:40.487831 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 18:19:40.496986 13023 raft_consensus.cc:1261] T 51b0cbfd37c44076afa99555f0e25f62 P 64fe9b685b894dee8ae689ccddf8b458 [term 1 FOLLOWER]: Refusing update from remote peer 9eb0135691d246f2944aaee5d68f999b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:19:40.497005 13223 raft_consensus.cc:1261] T 51b0cbfd37c44076afa99555f0e25f62 P fec595231b604f258412e1685da505d0 [term 1 FOLLOWER]: Refusing update from remote peer 9eb0135691d246f2944aaee5d68f999b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:19:40.498139 13338 consensus_queue.cc:1035] T 51b0cbfd37c44076afa99555f0e25f62 P 9eb0135691d246f2944aaee5d68f999b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fec595231b604f258412e1685da505d0" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 35251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:19:40.498540 13335 consensus_queue.cc:1035] T 51b0cbfd37c44076afa99555f0e25f62 P 9eb0135691d246f2944aaee5d68f999b [LEADER]: Connected to new peer: Peer: permanent_uuid: "64fe9b685b894dee8ae689ccddf8b458" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 40279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:19:40.511400 13344 mvcc.cc:205] Tried to move back new op lower bound from 7315975292904697856 to 7315975292771135488. Current Snapshot: MvccSnapshot[applied={T|T < 7315975292904697856}]
I20260807 18:19:40.511929 13347 mvcc.cc:205] Tried to move back new op lower bound from 7315975292904697856 to 7315975292771135488. Current Snapshot: MvccSnapshot[applied={T|T < 7315975292904697856}]
I20260807 18:19:40.516760 13345 mvcc.cc:205] Tried to move back new op lower bound from 7315975292904697856 to 7315975292771135488. Current Snapshot: MvccSnapshot[applied={T|T < 7315975292904697856}]
I20260807 18:19:40.527995 12620 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:19:40.528371 12620 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:19:40.533128 12620 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0c9af76d21f4cb4a47f1e3538983e71: Sending DeleteTablet for 3 replicas of tablet 51b0cbfd37c44076afa99555f0e25f62
I20260807 18:19:40.534588 13191 tablet_service.cc:1514] Processing DeleteTablet for tablet 51b0cbfd37c44076afa99555f0e25f62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:19:40 UTC) from {username='slave'} at 127.0.0.1:59930
I20260807 18:19:40.534610 12995 tablet_service.cc:1514] Processing DeleteTablet for tablet 51b0cbfd37c44076afa99555f0e25f62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:19:40 UTC) from {username='slave'} at 127.0.0.1:43974
I20260807 18:19:40.535487 12795 tablet_service.cc:1514] Processing DeleteTablet for tablet 51b0cbfd37c44076afa99555f0e25f62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:19:40 UTC) from {username='slave'} at 127.0.0.1:41856
I20260807 18:19:40.538340 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 12711
I20260807 18:19:40.545612 13356 tablet_replica.cc:330] T 51b0cbfd37c44076afa99555f0e25f62 P fec595231b604f258412e1685da505d0: stopping tablet replica
I20260807 18:19:40.546344 13359 tablet_replica.cc:330] T 51b0cbfd37c44076afa99555f0e25f62 P 9eb0135691d246f2944aaee5d68f999b: stopping tablet replica
I20260807 18:19:40.547035 13359 raft_consensus.cc:2229] T 51b0cbfd37c44076afa99555f0e25f62 P 9eb0135691d246f2944aaee5d68f999b [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:19:40.547590 13356 raft_consensus.cc:2229] T 51b0cbfd37c44076afa99555f0e25f62 P fec595231b604f258412e1685da505d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:19:40.547994 13359 raft_consensus.cc:2258] T 51b0cbfd37c44076afa99555f0e25f62 P 9eb0135691d246f2944aaee5d68f999b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:19:40.548244 13356 raft_consensus.cc:2258] T 51b0cbfd37c44076afa99555f0e25f62 P fec595231b604f258412e1685da505d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:19:40.549903 13357 tablet_replica.cc:330] T 51b0cbfd37c44076afa99555f0e25f62 P 64fe9b685b894dee8ae689ccddf8b458: stopping tablet replica
I20260807 18:19:40.550379 13357 raft_consensus.cc:2229] T 51b0cbfd37c44076afa99555f0e25f62 P 64fe9b685b894dee8ae689ccddf8b458 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:19:40.551348 13357 raft_consensus.cc:2258] T 51b0cbfd37c44076afa99555f0e25f62 P 64fe9b685b894dee8ae689ccddf8b458 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:19:40.553695 13357 ts_tablet_manager.cc:1905] T 51b0cbfd37c44076afa99555f0e25f62 P 64fe9b685b894dee8ae689ccddf8b458: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:19:40.553967 13356 ts_tablet_manager.cc:1905] T 51b0cbfd37c44076afa99555f0e25f62 P fec595231b604f258412e1685da505d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:19:40.554373 13359 ts_tablet_manager.cc:1905] T 51b0cbfd37c44076afa99555f0e25f62 P 9eb0135691d246f2944aaee5d68f999b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:19:40.558838 13323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:40.540130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41872 (local address 127.17.110.1:41941)
0807 18:19:40.540374 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:40.540383 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:40.546596 (+ 6213us) server_negotiation.cc:407] Connection header received
0807 18:19:40.546991 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:40.547001 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:40.547186 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:40.547267 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:40.548069 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.549569 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.552629 (+ 3060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.553271 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.553387 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:40.556862 (+ 3475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:40.556874 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:40.556892 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:19:40.557778 (+ 886us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:40.557849 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:19:40.557878 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:40.557970 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:40.558287 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 18:19:40.558416 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:19:40.560217 12887 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:19:40.565855 13357 ts_tablet_manager.cc:1918] T 51b0cbfd37c44076afa99555f0e25f62 P 64fe9b685b894dee8ae689ccddf8b458: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:19:40.566217 13357 log.cc:1196] T 51b0cbfd37c44076afa99555f0e25f62 P 64fe9b685b894dee8ae689ccddf8b458: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-1/wal/wals/51b0cbfd37c44076afa99555f0e25f62
I20260807 18:19:40.567133 13357 ts_tablet_manager.cc:1939] T 51b0cbfd37c44076afa99555f0e25f62 P 64fe9b685b894dee8ae689ccddf8b458: Deleting consensus metadata
I20260807 18:19:40.568959 13356 ts_tablet_manager.cc:1918] T 51b0cbfd37c44076afa99555f0e25f62 P fec595231b604f258412e1685da505d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:19:40.569187 13356 log.cc:1196] T 51b0cbfd37c44076afa99555f0e25f62 P fec595231b604f258412e1685da505d0: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126429568977-17848-0/minicluster-data/ts-2/wal/wals/51b0cbfd37c44076afa99555f0e25f62
I20260807 18:19:40.569461 12600 catalog_manager.cc:4841] TS 64fe9b685b894dee8ae689ccddf8b458 (127.17.110.2:40279): tablet 51b0cbfd37c44076afa99555f0e25f62 (table test-table [id=e04002a694c74e06af9f399d635569e6]) successfully deleted
I20260807 18:19:40.569823 13356 ts_tablet_manager.cc:1939] T 51b0cbfd37c44076afa99555f0e25f62 P fec595231b604f258412e1685da505d0: Deleting consensus metadata
I20260807 18:19:40.571705 12599 catalog_manager.cc:4841] TS fec595231b604f258412e1685da505d0 (127.17.110.3:35251): tablet 51b0cbfd37c44076afa99555f0e25f62 (table test-table [id=e04002a694c74e06af9f399d635569e6]) successfully deleted
I20260807 18:19:40.671176 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 12711
W20260807 18:19:40.698815 12597 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.110.1:41941 (error 108)
I20260807 18:19:40.699072 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 12900
I20260807 18:19:40.701151 13321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:40.700634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:41941 (local address 127.0.0.1:41886)
0807 18:19:40.700813 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:40.700962 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:41941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260807 18:19:40.701449 12597 catalog_manager.cc:4584] TS 9eb0135691d246f2944aaee5d68f999b (127.17.110.1:41941): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 51b0cbfd37c44076afa99555f0e25f62: Network error: Client connection negotiation failed: client connection to 127.17.110.1:41941: connect: Connection refused (error 111)
I20260807 18:19:40.706954 13322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:40.700033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43986 (local address 127.17.110.2:40279)
0807 18:19:40.700276 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:40.700281 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:40.700426 (+ 145us) server_negotiation.cc:407] Connection header received
0807 18:19:40.700608 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:40.700614 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:40.700759 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:40.700900 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:40.701502 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.702332 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.703751 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.704173 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.704314 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:40.705478 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:40.705487 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:40.705503 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:19:40.706107 (+ 604us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:40.706159 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:19:40.706179 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:40.706319 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:40.706458 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 18:19:40.706573 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260807 18:19:40.707661 13078 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:19:40.755381 13321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:40.754768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:41941 (local address 127.0.0.1:41894)
0807 18:19:40.755007 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:40.755169 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:41941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 18:19:40.756739 13321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:40.756335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:41941 (local address 127.0.0.1:41898)
0807 18:19:40.756503 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:40.756590 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:41941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 18:19:40.809482 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 12900
I20260807 18:19:40.819898 13321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:40.819306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:41941 (local address 127.0.0.1:41900)
0807 18:19:40.819527 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:40.819635 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:41941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 18:19:40.821274 13321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:40.820826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:41941 (local address 127.0.0.1:41902)
0807 18:19:40.820998 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:40.821093 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:41941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 18:19:40.843575 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 13092
I20260807 18:19:40.851261 13319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:40.844455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59940 (local address 127.17.110.3:35251)
0807 18:19:40.844947 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:40.844952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:40.844968 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:19:40.845023 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:40.845026 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:40.845107 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:40.845227 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:40.845658 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.846400 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.847816 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:40.848221 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:40.848380 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:40.849607 (+ 1227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:40.849619 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:40.849636 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:19:40.850289 (+ 653us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:40.850347 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:19:40.850370 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:40.850520 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:40.850694 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 18:19:40.850954 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377}
I20260807 18:19:40.852252 13289 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:19:40.902262 13321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:40.901659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:41941 (local address 127.0.0.1:41914)
0807 18:19:40.901951 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:40.902065 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:41941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
I20260807 18:19:40.903650 13321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:40.903143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:41941 (local address 127.0.0.1:41920)
0807 18:19:40.903411 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:40.903502 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:41941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
I20260807 18:19:40.946934 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 13092
I20260807 18:19:40.969803 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 12571
I20260807 18:19:40.971113 12663 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:19:41.085906 13321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:41.085133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:41941 (local address 127.0.0.1:41932)
0807 18:19:41.085475 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:41.085629 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:41941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160}
I20260807 18:19:41.085898 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 12571
2026-08-07T18:19:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3051 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:19:41 dist-test-slave-39l8 krb5kdc[13373](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:19:41 dist-test-slave-39l8 krb5kdc[13373](info): set up 2 sockets
Aug 07 18:19:41 dist-test-slave-39l8 krb5kdc[13373](info): commencing operation
krb5kdc: starting...
W20260807 18:19:42.454543 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.315s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:19:42 dist-test-slave-39l8 krb5kdc[13373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:19:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:19:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.110.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
I20260807 18:19:42.800185 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:34789
--webserver_interface=127.17.110.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:34875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:34789
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:19:43.176066 13399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:19:43.176403 13399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:19:43.176505 13399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:19:43.184196 13399 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:19:43.184329 13399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:19:43.184378 13399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:19:43.184428 13399 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:19:43.184475 13399 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:19:43.195974 13399 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:34875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:34789
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:34789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:19:43.197510 13399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:19:43.198901 13399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:19:43.211689 13405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:43.211774 13406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:43.211723 13409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:19:43.212674 13399 server_base.cc:1034] running on GCE node
I20260807 18:19:43.213591 13399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:19:43.215355 13399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:19:43.216614 13399 hybrid_clock.cc:648] HybridClock initialized: now 1786126783216558 us; error 51 us; skew 500 ppm
I20260807 18:19:43.217012 13399 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:19:43 dist-test-slave-39l8 krb5kdc[13373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126783, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:19:43.226822 13399 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:19:43.228858 13399 webserver.cc:460] Webserver started at http://127.17.110.62:32811/ using document root <none> and password file <none>
I20260807 18:19:43.229761 13399 fs_manager.cc:359] Metadata directory not provided
I20260807 18:19:43.229944 13399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:19:43.230319 13399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:19:43.234055 13399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "af3ccefde21b4d9e8d589ec862ef04c2"
format_stamp: "Formatted at 2026-08-07 18:19:43 on dist-test-slave-39l8"
server_key: "259eda50d403ebadbaffcc6023a99a69"
server_key_iv: "9dbb81d8630a4d8ffd7ab19245dd7313"
server_key_version: "encryptionkey@0"
I20260807 18:19:43.235116 13399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "af3ccefde21b4d9e8d589ec862ef04c2"
format_stamp: "Formatted at 2026-08-07 18:19:43 on dist-test-slave-39l8"
server_key: "259eda50d403ebadbaffcc6023a99a69"
server_key_iv: "9dbb81d8630a4d8ffd7ab19245dd7313"
server_key_version: "encryptionkey@0"
I20260807 18:19:43.241487 13399 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 18:19:43.245918 13417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:43.247730 13399 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:19:43.247985 13399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "af3ccefde21b4d9e8d589ec862ef04c2"
format_stamp: "Formatted at 2026-08-07 18:19:43 on dist-test-slave-39l8"
server_key: "259eda50d403ebadbaffcc6023a99a69"
server_key_iv: "9dbb81d8630a4d8ffd7ab19245dd7313"
server_key_version: "encryptionkey@0"
I20260807 18:19:43.248258 13399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:19:43.265604 13399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:19:43.271169 13399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:19:43.271519 13399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:19:43.293788 13399 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:34789
I20260807 18:19:43.293793 13492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:34789 every 8 connection(s)
I20260807 18:19:43.295526 13399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:19:43.296022 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 13399
I20260807 18:19:43.296219 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:19:43.296926 17848 external_mini_cluster.cc:1428] Setting key 0fb4f07afe29c18790d5e64a0983b043
I20260807 18:19:43.300503 13495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 18:19:43 dist-test-slave-39l8 krb5kdc[13373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:19:43.315837 13495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af3ccefde21b4d9e8d589ec862ef04c2: Bootstrap starting.
I20260807 18:19:43.320500 13495 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af3ccefde21b4d9e8d589ec862ef04c2: Neither blocks nor log segments found. Creating new log.
I20260807 18:19:43.321892 13495 log.cc:824] T 00000000000000000000000000000000 P af3ccefde21b4d9e8d589ec862ef04c2: Log is configured to *not* fsync() on all Append() calls
I20260807 18:19:43.325608 13499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:43.299112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37718 (local address 127.17.110.62:34789)
0807 18:19:43.300019 (+ 907us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:43.300033 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:43.300083 (+ 50us) server_negotiation.cc:407] Connection header received
0807 18:19:43.302042 (+ 1959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:43.302067 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:43.302504 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:43.302938 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:43.304936 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:43.306774 (+ 1838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:43.308848 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:43.309788 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:43.309954 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:43.315576 (+ 5622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:43.315591 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:43.315616 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:19:43.315674 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:43.319904 (+ 4230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:43.321073 (+ 1169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:43.321083 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:43.321093 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:43.321344 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:43.321858 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:43.321865 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:43.321873 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:43.322750 (+ 877us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:43.323007 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:43.323637 (+ 630us) server_negotiation.cc:299] Negotiation successful
0807 18:19:43.324136 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":430,"threads_started":1}
I20260807 18:19:43.325924 13495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af3ccefde21b4d9e8d589ec862ef04c2: No bootstrap required, opened a new log
I20260807 18:19:43.334455 13495 raft_consensus.cc:348] T 00000000000000000000000000000000 P af3ccefde21b4d9e8d589ec862ef04c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af3ccefde21b4d9e8d589ec862ef04c2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 34789 } }
I20260807 18:19:43.334987 13495 raft_consensus.cc:374] T 00000000000000000000000000000000 P af3ccefde21b4d9e8d589ec862ef04c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:19:43.335067 13495 raft_consensus.cc:729] T 00000000000000000000000000000000 P af3ccefde21b4d9e8d589ec862ef04c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af3ccefde21b4d9e8d589ec862ef04c2, State: Initialized, Role: FOLLOWER
I20260807 18:19:43.335608 13495 consensus_queue.cc:260] T 00000000000000000000000000000000 P af3ccefde21b4d9e8d589ec862ef04c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af3ccefde21b4d9e8d589ec862ef04c2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 34789 } }
I20260807 18:19:43.335840 13495 raft_consensus.cc:388] T 00000000000000000000000000000000 P af3ccefde21b4d9e8d589ec862ef04c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:19:43.335990 13495 raft_consensus.cc:482] T 00000000000000000000000000000000 P af3ccefde21b4d9e8d589ec862ef04c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:19:43.336179 13495 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af3ccefde21b4d9e8d589ec862ef04c2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:19:43.338932 13495 raft_consensus.cc:504] T 00000000000000000000000000000000 P af3ccefde21b4d9e8d589ec862ef04c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af3ccefde21b4d9e8d589ec862ef04c2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 34789 } }
I20260807 18:19:43.339481 13495 leader_election.cc:302] T 00000000000000000000000000000000 P af3ccefde21b4d9e8d589ec862ef04c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af3ccefde21b4d9e8d589ec862ef04c2; no voters:
I20260807 18:19:43.340106 13495 leader_election.cc:288] T 00000000000000000000000000000000 P af3ccefde21b4d9e8d589ec862ef04c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:19:43.340281 13502 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af3ccefde21b4d9e8d589ec862ef04c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:19:43.341040 13502 raft_consensus.cc:686] T 00000000000000000000000000000000 P af3ccefde21b4d9e8d589ec862ef04c2 [term 1 LEADER]: Becoming Leader. State: Replica: af3ccefde21b4d9e8d589ec862ef04c2, State: Running, Role: LEADER
I20260807 18:19:43.341717 13502 consensus_queue.cc:237] T 00000000000000000000000000000000 P af3ccefde21b4d9e8d589ec862ef04c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af3ccefde21b4d9e8d589ec862ef04c2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 34789 } }
I20260807 18:19:43.342509 13495 sys_catalog.cc:564] T 00000000000000000000000000000000 P af3ccefde21b4d9e8d589ec862ef04c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:19:43.345330 13504 sys_catalog.cc:455] T 00000000000000000000000000000000 P af3ccefde21b4d9e8d589ec862ef04c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af3ccefde21b4d9e8d589ec862ef04c2. Latest consensus state: current_term: 1 leader_uuid: "af3ccefde21b4d9e8d589ec862ef04c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af3ccefde21b4d9e8d589ec862ef04c2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 34789 } } }
I20260807 18:19:43.345296 13503 sys_catalog.cc:455] T 00000000000000000000000000000000 P af3ccefde21b4d9e8d589ec862ef04c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af3ccefde21b4d9e8d589ec862ef04c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af3ccefde21b4d9e8d589ec862ef04c2" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 34789 } } }
I20260807 18:19:43.346074 13504 sys_catalog.cc:458] T 00000000000000000000000000000000 P af3ccefde21b4d9e8d589ec862ef04c2 [sys.catalog]: This master's current role is: LEADER
I20260807 18:19:43.346079 13503 sys_catalog.cc:458] T 00000000000000000000000000000000 P af3ccefde21b4d9e8d589ec862ef04c2 [sys.catalog]: This master's current role is: LEADER
I20260807 18:19:43.348634 13514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:19:43.354245 13514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:19:43.365617 13514 catalog_manager.cc:1347] Generated new cluster ID: 9bb8fd031b414d4d828fd6d3897ffbc8
I20260807 18:19:43.365808 13514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:19:43.388234 13514 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:19:43.389715 13514 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:19:43.395874 13514 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af3ccefde21b4d9e8d589ec862ef04c2: Generated new TSK 0
I20260807 18:19:43.396986 13514 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.110.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
Entry for principal kudu/127.17.110.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab.
I20260807 18:19:43.494251 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.1:0
--local_ip_for_outbound_sockets=127.17.110.1
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:34789
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:34875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:19:43.861249 13530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:19:43.861562 13530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:19:43.861668 13530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:19:43.869114 13530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:19:43.869252 13530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:19:43.869369 13530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.1
I20260807 18:19:43.879817 13530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:34875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.1.keytab
--principal=kudu/127.17.110.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:34789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.1
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:19:43.881135 13530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:19:43.882421 13530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:19:43.896592 13543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:43.896543 13548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:43.896642 13545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:19:43.900317 13530 server_base.cc:1034] running on GCE node
I20260807 18:19:43.901119 13530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:19:43.902252 13530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:19:43.903517 13530 hybrid_clock.cc:648] HybridClock initialized: now 1786126783903462 us; error 64 us; skew 500 ppm
I20260807 18:19:43.903906 13530 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:19:43 dist-test-slave-39l8 krb5kdc[13373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126783, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:19:43.907752 13530 init.cc:377] Logged in from keytab as kudu/127.17.110.1@KRBTEST.COM (short username kudu)
I20260807 18:19:43.909505 13530 webserver.cc:460] Webserver started at http://127.17.110.1:37883/ using document root <none> and password file <none>
I20260807 18:19:43.910323 13530 fs_manager.cc:359] Metadata directory not provided
I20260807 18:19:43.910444 13530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:19:43.910838 13530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:19:43.914232 13530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-0/data/instance:
uuid: "e32f6f4c22594a90a800d40a6c9cfc8d"
format_stamp: "Formatted at 2026-08-07 18:19:43 on dist-test-slave-39l8"
server_key: "108ccb4e2630254d6b4028941e9fa46e"
server_key_iv: "8bea9d6270b286ad6124cdb9ac95623a"
server_key_version: "encryptionkey@0"
I20260807 18:19:43.915205 13530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance:
uuid: "e32f6f4c22594a90a800d40a6c9cfc8d"
format_stamp: "Formatted at 2026-08-07 18:19:43 on dist-test-slave-39l8"
server_key: "108ccb4e2630254d6b4028941e9fa46e"
server_key_iv: "8bea9d6270b286ad6124cdb9ac95623a"
server_key_version: "encryptionkey@0"
I20260807 18:19:43.921583 13530 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.005s
I20260807 18:19:43.926100 13558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:43.928052 13530 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260807 18:19:43.928275 13530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-0/wal
uuid: "e32f6f4c22594a90a800d40a6c9cfc8d"
format_stamp: "Formatted at 2026-08-07 18:19:43 on dist-test-slave-39l8"
server_key: "108ccb4e2630254d6b4028941e9fa46e"
server_key_iv: "8bea9d6270b286ad6124cdb9ac95623a"
server_key_version: "encryptionkey@0"
I20260807 18:19:43.928530 13530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:19:43.945390 13530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:19:43.950683 13530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:19:43.951095 13530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:19:43.952244 13530 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:19:43.953876 13530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:19:43.953955 13530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:43.954088 13530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:19:43.954165 13530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:43.991215 13530 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.1:41945
I20260807 18:19:43.991235 13736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.1:41945 every 8 connection(s)
I20260807 18:19:43.992640 13530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
I20260807 18:19:43.999555 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 13530
I20260807 18:19:43.999770 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance
I20260807 18:19:44.000238 17848 external_mini_cluster.cc:1428] Setting key 3aa6e1640c1a0f67416a02be34b58e44
Aug 07 18:19:44 dist-test-slave-39l8 krb5kdc[13373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126783, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.1@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:19:44.020679 13740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:43.996175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:34789 (local address 127.17.110.1:48515)
0807 18:19:43.996976 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:43.997029 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:43.999044 (+ 2015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:43.999963 (+ 919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:43.999972 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:44.000653 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:19:44.001478 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:44.001501 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:44.002986 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:44.002991 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:44.004553 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:44.004571 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:44.005518 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:44.005530 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:44.005808 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:44.006634 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:19:44.006668 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:19:44.009802 (+ 3134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:19:44.014542 (+ 4740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:19:44.014555 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:19:44.014596 (+ 41us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:19:44.015534 (+ 938us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:19:44.016249 (+ 715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:19:44.016255 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:19:44.016259 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:19:44.016636 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:19:44.017552 (+ 916us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:19:44.017562 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:19:44.018167 (+ 605us) client_negotiation.cc:769] Sending connection context
0807 18:19:44.018583 (+ 416us) client_negotiation.cc:240] Negotiation successful
0807 18:19:44.019192 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":291,"threads_started":1}
I20260807 18:19:44.022034 13739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:43.995592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:48515 (local address 127.17.110.62:34789)
0807 18:19:43.996145 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:43.996159 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:43.997173 (+ 1014us) server_negotiation.cc:407] Connection header received
0807 18:19:43.999404 (+ 2231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:43.999411 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:43.999515 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:43.999645 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:44.001694 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:44.002768 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:44.004753 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:44.005225 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:44.005390 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:44.010102 (+ 4712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:44.010108 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:44.010112 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:19:44.010153 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:44.014267 (+ 4114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:44.015734 (+ 1467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:44.015741 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:44.015745 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:44.016015 (+ 270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:44.016826 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:44.016834 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:44.016838 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:44.017272 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:44.017451 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:44.021522 (+ 4071us) server_negotiation.cc:299] Negotiation successful
0807 18:19:44.021752 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":269,"threads_started":1}
I20260807 18:19:44.023936 13737 heartbeater.cc:347] Connected to a master server at 127.17.110.62:34789
I20260807 18:19:44.024336 13737 heartbeater.cc:464] Registering TS with master...
I20260807 18:19:44.025240 13737 heartbeater.cc:511] Master 127.17.110.62:34789 requested a full tablet report, sending...
I20260807 18:19:44.027819 13439 ts_manager.cc:194] Registered new tserver with Master: e32f6f4c22594a90a800d40a6c9cfc8d (127.17.110.1:41945)
I20260807 18:19:44.029485 13439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.1@KRBTEST.COM'} at 127.17.110.1:48515
WARNING: no policy specified for kudu/127.17.110.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
Entry for principal kudu/127.17.110.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab.
I20260807 18:19:44.082880 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.2:0
--local_ip_for_outbound_sockets=127.17.110.2
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:34789
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:34875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:19:44.458604 13744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:19:44.459019 13744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:19:44.459195 13744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:19:44.467774 13744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:19:44.467952 13744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:19:44.468182 13744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.2
I20260807 18:19:44.479111 13744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:34875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.2.keytab
--principal=kudu/127.17.110.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:34789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.2
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:19:44.480530 13744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:19:44.481911 13744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:19:44.496443 13752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:44.496358 13755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:44.496402 13753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:19:44.497210 13744 server_base.cc:1034] running on GCE node
I20260807 18:19:44.497926 13744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:19:44.498983 13744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:19:44.500253 13744 hybrid_clock.cc:648] HybridClock initialized: now 1786126784500191 us; error 54 us; skew 500 ppm
I20260807 18:19:44.500648 13744 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:19:44 dist-test-slave-39l8 krb5kdc[13373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126784, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:19:44.504323 13744 init.cc:377] Logged in from keytab as kudu/127.17.110.2@KRBTEST.COM (short username kudu)
I20260807 18:19:44.506016 13744 webserver.cc:460] Webserver started at http://127.17.110.2:37411/ using document root <none> and password file <none>
I20260807 18:19:44.506884 13744 fs_manager.cc:359] Metadata directory not provided
I20260807 18:19:44.507037 13744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:19:44.507393 13744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:19:44.510823 13744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-1/data/instance:
uuid: "6c46c18ff6384e02a65d5c945a90b15e"
format_stamp: "Formatted at 2026-08-07 18:19:44 on dist-test-slave-39l8"
server_key: "1d8d0a4d6faf82f10c4d0c0c4f162554"
server_key_iv: "2f8710f027d364e63ac55909ad9b77ff"
server_key_version: "encryptionkey@0"
I20260807 18:19:44.511689 13744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance:
uuid: "6c46c18ff6384e02a65d5c945a90b15e"
format_stamp: "Formatted at 2026-08-07 18:19:44 on dist-test-slave-39l8"
server_key: "1d8d0a4d6faf82f10c4d0c0c4f162554"
server_key_iv: "2f8710f027d364e63ac55909ad9b77ff"
server_key_version: "encryptionkey@0"
I20260807 18:19:44.517400 13744 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 18:19:44.521611 13766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:44.523561 13744 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:19:44.523799 13744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-1/wal
uuid: "6c46c18ff6384e02a65d5c945a90b15e"
format_stamp: "Formatted at 2026-08-07 18:19:44 on dist-test-slave-39l8"
server_key: "1d8d0a4d6faf82f10c4d0c0c4f162554"
server_key_iv: "2f8710f027d364e63ac55909ad9b77ff"
server_key_version: "encryptionkey@0"
I20260807 18:19:44.524129 13744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:19:44.547921 13744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:19:44.552964 13744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:19:44.553368 13744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:19:44.554567 13744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:19:44.556227 13744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:19:44.556346 13744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:44.556510 13744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:19:44.556591 13744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:44.602427 13744 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.2:39429
I20260807 18:19:44.602519 13923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.2:39429 every 8 connection(s)
I20260807 18:19:44.603879 13744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
I20260807 18:19:44.612478 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 13744
I20260807 18:19:44.612712 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance
I20260807 18:19:44.613190 17848 external_mini_cluster.cc:1428] Setting key 37a720674585a8db26672626653c0f7e
Aug 07 18:19:44 dist-test-slave-39l8 krb5kdc[13373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126784, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.2@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:19:44.628216 13929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:44.606663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:54361 (local address 127.17.110.62:34789)
0807 18:19:44.607202 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:44.607207 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:44.608130 (+ 923us) server_negotiation.cc:407] Connection header received
0807 18:19:44.610183 (+ 2053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:44.610189 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:44.610293 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:44.610417 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:44.612136 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:44.612936 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:44.614977 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:44.615606 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:44.615746 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:44.619767 (+ 4021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:44.619777 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:44.619785 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:19:44.619835 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:44.623665 (+ 3830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:44.624919 (+ 1254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:44.624928 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:44.624932 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:44.625155 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:44.625724 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:44.625733 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:44.625739 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:44.626206 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:44.626381 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:44.627410 (+ 1029us) server_negotiation.cc:299] Negotiation successful
0807 18:19:44.627706 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":263,"threads_started":1}
I20260807 18:19:44.629021 13930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:44.607192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:34789 (local address 127.17.110.2:54361)
0807 18:19:44.607954 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:44.607993 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:44.609875 (+ 1882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:44.610684 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:44.610692 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:44.611319 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:19:44.611917 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:44.611941 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:44.613109 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:44.613113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:44.614821 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:44.614837 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:44.615836 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:44.615841 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:44.616061 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:44.616834 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:19:44.616859 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:19:44.619445 (+ 2586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:19:44.623892 (+ 4447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:19:44.623902 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:19:44.623919 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:19:44.624580 (+ 661us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:19:44.625326 (+ 746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:19:44.625331 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:19:44.625333 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:19:44.625571 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:19:44.626382 (+ 811us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:19:44.626391 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:19:44.626936 (+ 545us) client_negotiation.cc:769] Sending connection context
0807 18:19:44.627211 (+ 275us) client_negotiation.cc:240] Negotiation successful
0807 18:19:44.627680 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":314,"threads_started":1}
I20260807 18:19:44.630857 13924 heartbeater.cc:347] Connected to a master server at 127.17.110.62:34789
I20260807 18:19:44.631258 13924 heartbeater.cc:464] Registering TS with master...
I20260807 18:19:44.632056 13924 heartbeater.cc:511] Master 127.17.110.62:34789 requested a full tablet report, sending...
I20260807 18:19:44.634001 13439 ts_manager.cc:194] Registered new tserver with Master: 6c46c18ff6384e02a65d5c945a90b15e (127.17.110.2:39429)
I20260807 18:19:44.635079 13439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.2@KRBTEST.COM'} at 127.17.110.2:54361
WARNING: no policy specified for kudu/127.17.110.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
Entry for principal kudu/127.17.110.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab.
I20260807 18:19:44.689877 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.3:0
--local_ip_for_outbound_sockets=127.17.110.3
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:34789
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:34875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:19:45.033242 13737 heartbeater.cc:503] Master 127.17.110.62:34789 was elected leader, sending a full tablet report...
W20260807 18:19:45.046672 13934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:19:45.047049 13934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:19:45.047160 13934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:19:45.054960 13934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:19:45.055092 13934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:19:45.055217 13934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.3
I20260807 18:19:45.065387 13934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:34875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.3.keytab
--principal=kudu/127.17.110.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:34789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.3
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:19:45.066785 13934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:19:45.068102 13934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:19:45.081776 13945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:45.081833 13942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:45.081832 13947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:19:45.083283 13934 server_base.cc:1034] running on GCE node
I20260807 18:19:45.083968 13934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:19:45.085016 13934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:19:45.086272 13934 hybrid_clock.cc:648] HybridClock initialized: now 1786126785086206 us; error 69 us; skew 500 ppm
I20260807 18:19:45.086608 13934 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:19:45 dist-test-slave-39l8 krb5kdc[13373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126785, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:19:45.090121 13934 init.cc:377] Logged in from keytab as kudu/127.17.110.3@KRBTEST.COM (short username kudu)
I20260807 18:19:45.091922 13934 webserver.cc:460] Webserver started at http://127.17.110.3:41649/ using document root <none> and password file <none>
I20260807 18:19:45.092703 13934 fs_manager.cc:359] Metadata directory not provided
I20260807 18:19:45.092860 13934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:19:45.093196 13934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:19:45.096367 13934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-2/data/instance:
uuid: "3bdd3c39149842e5bd679e06e4609ce4"
format_stamp: "Formatted at 2026-08-07 18:19:45 on dist-test-slave-39l8"
server_key: "aa7b6593df9db583fd07a7a596663a97"
server_key_iv: "e492903376956c27a13251b78593007b"
server_key_version: "encryptionkey@0"
I20260807 18:19:45.097198 13934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance:
uuid: "3bdd3c39149842e5bd679e06e4609ce4"
format_stamp: "Formatted at 2026-08-07 18:19:45 on dist-test-slave-39l8"
server_key: "aa7b6593df9db583fd07a7a596663a97"
server_key_iv: "e492903376956c27a13251b78593007b"
server_key_version: "encryptionkey@0"
I20260807 18:19:45.102959 13934 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260807 18:19:45.106910 13957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:45.108575 13934 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:19:45.108781 13934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-2/wal
uuid: "3bdd3c39149842e5bd679e06e4609ce4"
format_stamp: "Formatted at 2026-08-07 18:19:45 on dist-test-slave-39l8"
server_key: "aa7b6593df9db583fd07a7a596663a97"
server_key_iv: "e492903376956c27a13251b78593007b"
server_key_version: "encryptionkey@0"
I20260807 18:19:45.108989 13934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:19:45.140944 13934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:19:45.146051 13934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:19:45.146376 13934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:19:45.147545 13934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:19:45.149087 13934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:19:45.149166 13934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:45.149267 13934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:19:45.149314 13934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:45.185837 13934 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.3:35821
I20260807 18:19:45.185886 14135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.3:35821 every 8 connection(s)
I20260807 18:19:45.187314 13934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
I20260807 18:19:45.195871 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 13934
I20260807 18:19:45.196095 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance
I20260807 18:19:45.196614 17848 external_mini_cluster.cc:1428] Setting key 80514fb9f5b79fa9d72d8d8fbc4c10bd
Aug 07 18:19:45 dist-test-slave-39l8 krb5kdc[13373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126785, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.3@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:19:45.209260 14138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:45.190077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:38145 (local address 127.17.110.62:34789)
0807 18:19:45.190629 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:45.190634 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:45.191508 (+ 874us) server_negotiation.cc:407] Connection header received
0807 18:19:45.193527 (+ 2019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:45.193533 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:45.193646 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:45.193772 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:45.195515 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.196407 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.198470 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.198967 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.199123 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:45.202909 (+ 3786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:45.202915 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:45.202919 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:19:45.202957 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:45.205831 (+ 2874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:45.206737 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:45.206744 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:45.206752 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:45.206884 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:45.207358 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:45.207362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:45.207365 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:45.207656 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:45.207828 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:45.208578 (+ 750us) server_negotiation.cc:299] Negotiation successful
0807 18:19:45.208821 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":287,"threads_started":1}
I20260807 18:19:45.210414 14139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:45.190516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:34789 (local address 127.17.110.3:38145)
0807 18:19:45.191298 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:45.191363 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:45.193218 (+ 1855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:45.194032 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:45.194041 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:45.194630 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:19:45.195316 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:45.195337 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.196578 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.196582 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:45.198306 (+ 1724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:45.198321 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.199229 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.199240 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:45.199487 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:45.200205 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:19:45.200234 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:19:45.202616 (+ 2382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:19:45.206023 (+ 3407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:19:45.206035 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:19:45.206056 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:19:45.206538 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:19:45.207007 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:19:45.207012 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:19:45.207016 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:19:45.207227 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:19:45.207792 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:19:45.207802 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:19:45.208192 (+ 390us) client_negotiation.cc:769] Sending connection context
0807 18:19:45.208474 (+ 282us) client_negotiation.cc:240] Negotiation successful
0807 18:19:45.209065 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":313,"threads_started":1}
I20260807 18:19:45.212189 14136 heartbeater.cc:347] Connected to a master server at 127.17.110.62:34789
I20260807 18:19:45.212572 14136 heartbeater.cc:464] Registering TS with master...
I20260807 18:19:45.213451 14136 heartbeater.cc:511] Master 127.17.110.62:34789 requested a full tablet report, sending...
I20260807 18:19:45.215040 13439 ts_manager.cc:194] Registered new tserver with Master: 3bdd3c39149842e5bd679e06e4609ce4 (127.17.110.3:35821)
I20260807 18:19:45.215850 13439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.110.3@KRBTEST.COM'} at 127.17.110.3:38145
I20260807 18:19:45.225721 17848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:19:45.244184 14138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:45.230409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37720 (local address 127.17.110.62:34789)
0807 18:19:45.230600 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:45.230605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:45.230994 (+ 389us) server_negotiation.cc:407] Connection header received
0807 18:19:45.231147 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:45.231151 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:45.231263 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:45.231409 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:45.233061 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.233892 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.235287 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.235707 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.235808 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:45.238276 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:45.238282 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:45.238285 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:19:45.238322 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:45.241259 (+ 2937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:45.242031 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:45.242036 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:45.242039 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:45.242197 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:45.242546 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:45.242550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:45.242552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:45.243068 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:45.243279 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:45.243675 (+ 396us) server_negotiation.cc:299] Negotiation successful
0807 18:19:45.243930 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 18:19:45.249054 13439 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37720:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:19:45.256496 13439 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:19:45.282409 14153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:45.270991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36922 (local address 127.17.110.2:39429)
0807 18:19:45.271800 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:45.271813 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:45.272570 (+ 757us) server_negotiation.cc:407] Connection header received
0807 18:19:45.273025 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:45.273033 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:45.273324 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:45.273498 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:19:45.274518 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.276884 (+ 2366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.279586 (+ 2702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.280850 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.281017 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:45.281265 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:45.281850 (+ 585us) server_negotiation.cc:299] Negotiation successful
0807 18:19:45.282029 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":371,"threads_started":1}
I20260807 18:19:45.282574 14152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:45.270304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:39429 (local address 127.0.0.1:36922)
0807 18:19:45.272411 (+ 2107us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:45.272453 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:45.272761 (+ 308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:45.273839 (+ 1078us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:45.273851 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:45.273879 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:19:45.274319 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:45.274334 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.277059 (+ 2725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.277065 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:45.279412 (+ 2347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:45.279427 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.281393 (+ 1966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.281400 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:45.281595 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:45.281638 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 18:19:45.281810 (+ 172us) client_negotiation.cc:240] Negotiation successful
0807 18:19:45.282021 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1874,"spinlock_wait_cycles":3968,"thread_start_us":1721,"threads_started":1}
I20260807 18:19:45.283367 14150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:45.269962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:41945 (local address 127.0.0.1:58466)
0807 18:19:45.272539 (+ 2577us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:45.272565 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:45.272772 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:45.275560 (+ 2788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:45.275570 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:45.275595 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:19:45.276022 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:45.276038 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.277914 (+ 1876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.277920 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:45.280223 (+ 2303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:45.280233 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.282279 (+ 2046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.282287 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:45.282615 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:45.282648 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 18:19:45.282769 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 18:19:45.282913 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1841,"mutex_wait_us":96,"spinlock_wait_cycles":42752,"thread_start_us":501,"threads_started":1}
I20260807 18:19:45.283799 14154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:45.272384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:35821 (local address 127.0.0.1:44742)
0807 18:19:45.273377 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:45.273398 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:45.273527 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:45.275089 (+ 1562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:45.275093 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:45.275118 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:19:45.275586 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:45.275597 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.277515 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.277520 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:45.280206 (+ 2686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:45.280222 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.282291 (+ 2069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.282298 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:45.283165 (+ 867us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:45.283196 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 18:19:45.283264 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 18:19:45.283360 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":841,"mutex_wait_us":621,"thread_start_us":877,"threads_started":1}
I20260807 18:19:45.284332 14151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:45.270322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58466 (local address 127.17.110.1:41945)
0807 18:19:45.274911 (+ 4589us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:45.274921 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:45.274951 (+ 30us) server_negotiation.cc:407] Connection header received
0807 18:19:45.275042 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:45.275050 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:45.275294 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:45.275566 (+ 272us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:19:45.276223 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.277758 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.280414 (+ 2656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.281952 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.283454 (+ 1502us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:45.283714 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:45.283800 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 18:19:45.283995 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4409,"thread_start_us":4186,"threads_started":1}
I20260807 18:19:45.285861 14155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:45.272991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44742 (local address 127.17.110.3:35821)
0807 18:19:45.274536 (+ 1545us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:45.274545 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:45.274577 (+ 32us) server_negotiation.cc:407] Connection header received
0807 18:19:45.274670 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:45.274677 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:45.274908 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:45.275097 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:19:45.275884 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.277348 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.280438 (+ 3090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.281952 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.282610 (+ 658us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:45.283854 (+ 1244us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:45.283949 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 18:19:45.285502 (+ 1553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1007,"thread_start_us":1168,"threads_started":1}
I20260807 18:19:45.286810 13833 tablet_service.cc:1467] Processing CreateTablet for tablet 428a5997c83741a38eab94f3f245b137 (DEFAULT_TABLE table=test-table [id=48c65df43bf24fba9978b57882ca74ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:19:45.287951 13638 tablet_service.cc:1467] Processing CreateTablet for tablet 428a5997c83741a38eab94f3f245b137 (DEFAULT_TABLE table=test-table [id=48c65df43bf24fba9978b57882ca74ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:19:45.288407 13833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 428a5997c83741a38eab94f3f245b137. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:19:45.289103 13638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 428a5997c83741a38eab94f3f245b137. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:19:45.289332 14033 tablet_service.cc:1467] Processing CreateTablet for tablet 428a5997c83741a38eab94f3f245b137 (DEFAULT_TABLE table=test-table [id=48c65df43bf24fba9978b57882ca74ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:19:45.290766 14033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 428a5997c83741a38eab94f3f245b137. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:19:45.310005 14157 tablet_bootstrap.cc:492] T 428a5997c83741a38eab94f3f245b137 P e32f6f4c22594a90a800d40a6c9cfc8d: Bootstrap starting.
I20260807 18:19:45.310110 14156 tablet_bootstrap.cc:492] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e: Bootstrap starting.
I20260807 18:19:45.316220 14157 tablet_bootstrap.cc:654] T 428a5997c83741a38eab94f3f245b137 P e32f6f4c22594a90a800d40a6c9cfc8d: Neither blocks nor log segments found. Creating new log.
I20260807 18:19:45.316462 14156 tablet_bootstrap.cc:654] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e: Neither blocks nor log segments found. Creating new log.
I20260807 18:19:45.317703 14157 log.cc:824] T 428a5997c83741a38eab94f3f245b137 P e32f6f4c22594a90a800d40a6c9cfc8d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:19:45.317869 14158 tablet_bootstrap.cc:492] T 428a5997c83741a38eab94f3f245b137 P 3bdd3c39149842e5bd679e06e4609ce4: Bootstrap starting.
I20260807 18:19:45.317947 14156 log.cc:824] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e: Log is configured to *not* fsync() on all Append() calls
I20260807 18:19:45.324100 14158 tablet_bootstrap.cc:654] T 428a5997c83741a38eab94f3f245b137 P 3bdd3c39149842e5bd679e06e4609ce4: Neither blocks nor log segments found. Creating new log.
I20260807 18:19:45.324963 14156 tablet_bootstrap.cc:492] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e: No bootstrap required, opened a new log
I20260807 18:19:45.324962 14157 tablet_bootstrap.cc:492] T 428a5997c83741a38eab94f3f245b137 P e32f6f4c22594a90a800d40a6c9cfc8d: No bootstrap required, opened a new log
I20260807 18:19:45.325296 14156 ts_tablet_manager.cc:1397] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.000s
I20260807 18:19:45.325363 14157 ts_tablet_manager.cc:1397] T 428a5997c83741a38eab94f3f245b137 P e32f6f4c22594a90a800d40a6c9cfc8d: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.004s
I20260807 18:19:45.325320 14158 log.cc:824] T 428a5997c83741a38eab94f3f245b137 P 3bdd3c39149842e5bd679e06e4609ce4: Log is configured to *not* fsync() on all Append() calls
I20260807 18:19:45.329056 14158 tablet_bootstrap.cc:492] T 428a5997c83741a38eab94f3f245b137 P 3bdd3c39149842e5bd679e06e4609ce4: No bootstrap required, opened a new log
I20260807 18:19:45.329388 14158 ts_tablet_manager.cc:1397] T 428a5997c83741a38eab94f3f245b137 P 3bdd3c39149842e5bd679e06e4609ce4: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260807 18:19:45.336781 14156 raft_consensus.cc:348] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c46c18ff6384e02a65d5c945a90b15e" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 39429 } } peers { permanent_uuid: "e32f6f4c22594a90a800d40a6c9cfc8d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 41945 } } peers { permanent_uuid: "3bdd3c39149842e5bd679e06e4609ce4" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 35821 } }
I20260807 18:19:45.337291 14156 raft_consensus.cc:374] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:19:45.337381 14156 raft_consensus.cc:729] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c46c18ff6384e02a65d5c945a90b15e, State: Initialized, Role: FOLLOWER
I20260807 18:19:45.338157 14156 consensus_queue.cc:260] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c46c18ff6384e02a65d5c945a90b15e" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 39429 } } peers { permanent_uuid: "e32f6f4c22594a90a800d40a6c9cfc8d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 41945 } } peers { permanent_uuid: "3bdd3c39149842e5bd679e06e4609ce4" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 35821 } }
I20260807 18:19:45.339778 13924 heartbeater.cc:503] Master 127.17.110.62:34789 was elected leader, sending a full tablet report...
I20260807 18:19:45.340564 14156 ts_tablet_manager.cc:1428] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e: Time spent starting tablet: real 0.015s user 0.010s sys 0.006s
I20260807 18:19:45.340651 14158 raft_consensus.cc:348] T 428a5997c83741a38eab94f3f245b137 P 3bdd3c39149842e5bd679e06e4609ce4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c46c18ff6384e02a65d5c945a90b15e" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 39429 } } peers { permanent_uuid: "e32f6f4c22594a90a800d40a6c9cfc8d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 41945 } } peers { permanent_uuid: "3bdd3c39149842e5bd679e06e4609ce4" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 35821 } }
I20260807 18:19:45.341135 14158 raft_consensus.cc:374] T 428a5997c83741a38eab94f3f245b137 P 3bdd3c39149842e5bd679e06e4609ce4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:19:45.341264 14158 raft_consensus.cc:729] T 428a5997c83741a38eab94f3f245b137 P 3bdd3c39149842e5bd679e06e4609ce4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bdd3c39149842e5bd679e06e4609ce4, State: Initialized, Role: FOLLOWER
I20260807 18:19:45.342041 14158 consensus_queue.cc:260] T 428a5997c83741a38eab94f3f245b137 P 3bdd3c39149842e5bd679e06e4609ce4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c46c18ff6384e02a65d5c945a90b15e" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 39429 } } peers { permanent_uuid: "e32f6f4c22594a90a800d40a6c9cfc8d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 41945 } } peers { permanent_uuid: "3bdd3c39149842e5bd679e06e4609ce4" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 35821 } }
I20260807 18:19:45.342268 14157 raft_consensus.cc:348] T 428a5997c83741a38eab94f3f245b137 P e32f6f4c22594a90a800d40a6c9cfc8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c46c18ff6384e02a65d5c945a90b15e" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 39429 } } peers { permanent_uuid: "e32f6f4c22594a90a800d40a6c9cfc8d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 41945 } } peers { permanent_uuid: "3bdd3c39149842e5bd679e06e4609ce4" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 35821 } }
I20260807 18:19:45.343101 14157 raft_consensus.cc:374] T 428a5997c83741a38eab94f3f245b137 P e32f6f4c22594a90a800d40a6c9cfc8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:19:45.343281 14157 raft_consensus.cc:729] T 428a5997c83741a38eab94f3f245b137 P e32f6f4c22594a90a800d40a6c9cfc8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e32f6f4c22594a90a800d40a6c9cfc8d, State: Initialized, Role: FOLLOWER
I20260807 18:19:45.344280 14157 consensus_queue.cc:260] T 428a5997c83741a38eab94f3f245b137 P e32f6f4c22594a90a800d40a6c9cfc8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c46c18ff6384e02a65d5c945a90b15e" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 39429 } } peers { permanent_uuid: "e32f6f4c22594a90a800d40a6c9cfc8d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 41945 } } peers { permanent_uuid: "3bdd3c39149842e5bd679e06e4609ce4" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 35821 } }
I20260807 18:19:45.347662 14136 heartbeater.cc:503] Master 127.17.110.62:34789 was elected leader, sending a full tablet report...
I20260807 18:19:45.348605 14158 ts_tablet_manager.cc:1428] T 428a5997c83741a38eab94f3f245b137 P 3bdd3c39149842e5bd679e06e4609ce4: Time spent starting tablet: real 0.019s user 0.007s sys 0.011s
I20260807 18:19:45.348958 14157 ts_tablet_manager.cc:1428] T 428a5997c83741a38eab94f3f245b137 P e32f6f4c22594a90a800d40a6c9cfc8d: Time spent starting tablet: real 0.023s user 0.018s sys 0.000s
W20260807 18:19:45.355950 13928 tablet.cc:2369] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:19:45.439108 14137 tablet.cc:2369] T 428a5997c83741a38eab94f3f245b137 P 3bdd3c39149842e5bd679e06e4609ce4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:19:45.495388 13738 tablet.cc:2369] T 428a5997c83741a38eab94f3f245b137 P e32f6f4c22594a90a800d40a6c9cfc8d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:19:45.506745 14166 raft_consensus.cc:482] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:19:45.507079 14166 raft_consensus.cc:504] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c46c18ff6384e02a65d5c945a90b15e" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 39429 } } peers { permanent_uuid: "e32f6f4c22594a90a800d40a6c9cfc8d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 41945 } } peers { permanent_uuid: "3bdd3c39149842e5bd679e06e4609ce4" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 35821 } }
I20260807 18:19:45.508919 14166 leader_election.cc:288] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e32f6f4c22594a90a800d40a6c9cfc8d (127.17.110.1:41945), 3bdd3c39149842e5bd679e06e4609ce4 (127.17.110.3:35821)
I20260807 18:19:45.516770 14155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:45.509517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:48737 (local address 127.17.110.3:35821)
0807 18:19:45.509733 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:45.509742 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:45.510423 (+ 681us) server_negotiation.cc:407] Connection header received
0807 18:19:45.510642 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:45.510649 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:45.510761 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:45.510840 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:19:45.511651 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.512522 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.514640 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.515737 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.515917 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:45.515999 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:45.516376 (+ 377us) server_negotiation.cc:299] Negotiation successful
0807 18:19:45.516504 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 18:19:45.516965 14169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:45.509312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:35821 (local address 127.17.110.2:48737)
0807 18:19:45.510266 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:45.510298 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:45.510480 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:45.510914 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:45.510919 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:45.510944 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:19:45.511500 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:45.511512 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.512707 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.512716 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:45.514486 (+ 1770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:45.514499 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.515961 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.515968 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:45.516148 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:45.516183 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 18:19:45.516348 (+ 165us) client_negotiation.cc:240] Negotiation successful
0807 18:19:45.516493 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"mutex_wait_us":69,"spinlock_wait_cycles":27520,"thread_start_us":433,"threads_started":1}
I20260807 18:19:45.517364 14151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:45.509518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:56803 (local address 127.17.110.1:41945)
0807 18:19:45.509746 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:45.509753 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:45.510519 (+ 766us) server_negotiation.cc:407] Connection header received
0807 18:19:45.510856 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:45.510865 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:45.510949 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:45.511053 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:19:45.511796 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.512620 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.515105 (+ 2485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.516285 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.516444 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:45.516496 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:45.517006 (+ 510us) server_negotiation.cc:299] Negotiation successful
0807 18:19:45.517140 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260807 18:19:45.518029 14171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:45.509513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:41945 (local address 127.17.110.2:56803)
0807 18:19:45.510361 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:45.510396 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:45.510553 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:45.511080 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:45.511083 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:45.511118 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:19:45.511604 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:45.511624 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.512769 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.512776 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:45.514510 (+ 1734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:45.514523 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.516642 (+ 2119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.516654 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:45.516792 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:45.516814 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 18:19:45.517633 (+ 819us) client_negotiation.cc:240] Negotiation successful
0807 18:19:45.517770 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":747,"mutex_wait_us":39,"thread_start_us":542,"threads_started":1}
I20260807 18:19:45.518146 14060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "428a5997c83741a38eab94f3f245b137" candidate_uuid: "6c46c18ff6384e02a65d5c945a90b15e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bdd3c39149842e5bd679e06e4609ce4" is_pre_election: true
I20260807 18:19:45.518872 14060 raft_consensus.cc:2454] T 428a5997c83741a38eab94f3f245b137 P 3bdd3c39149842e5bd679e06e4609ce4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c46c18ff6384e02a65d5c945a90b15e in term 0.
I20260807 18:19:45.519345 13669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "428a5997c83741a38eab94f3f245b137" candidate_uuid: "6c46c18ff6384e02a65d5c945a90b15e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e32f6f4c22594a90a800d40a6c9cfc8d" is_pre_election: true
I20260807 18:19:45.519554 13771 leader_election.cc:302] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bdd3c39149842e5bd679e06e4609ce4, 6c46c18ff6384e02a65d5c945a90b15e; no voters:
I20260807 18:19:45.519899 13669 raft_consensus.cc:2454] T 428a5997c83741a38eab94f3f245b137 P e32f6f4c22594a90a800d40a6c9cfc8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c46c18ff6384e02a65d5c945a90b15e in term 0.
I20260807 18:19:45.519948 14166 raft_consensus.cc:2781] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:19:45.520099 14166 raft_consensus.cc:482] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:19:45.520197 14166 raft_consensus.cc:3037] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:19:45.522440 14166 raft_consensus.cc:504] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c46c18ff6384e02a65d5c945a90b15e" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 39429 } } peers { permanent_uuid: "e32f6f4c22594a90a800d40a6c9cfc8d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 41945 } } peers { permanent_uuid: "3bdd3c39149842e5bd679e06e4609ce4" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 35821 } }
I20260807 18:19:45.523216 14166 leader_election.cc:288] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e [CANDIDATE]: Term 1 election: Requested vote from peers e32f6f4c22594a90a800d40a6c9cfc8d (127.17.110.1:41945), 3bdd3c39149842e5bd679e06e4609ce4 (127.17.110.3:35821)
I20260807 18:19:45.523733 14060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "428a5997c83741a38eab94f3f245b137" candidate_uuid: "6c46c18ff6384e02a65d5c945a90b15e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bdd3c39149842e5bd679e06e4609ce4"
I20260807 18:19:45.523733 13669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "428a5997c83741a38eab94f3f245b137" candidate_uuid: "6c46c18ff6384e02a65d5c945a90b15e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e32f6f4c22594a90a800d40a6c9cfc8d"
I20260807 18:19:45.523940 14060 raft_consensus.cc:3037] T 428a5997c83741a38eab94f3f245b137 P 3bdd3c39149842e5bd679e06e4609ce4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:19:45.523953 13669 raft_consensus.cc:3037] T 428a5997c83741a38eab94f3f245b137 P e32f6f4c22594a90a800d40a6c9cfc8d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:19:45.526254 14060 raft_consensus.cc:2454] T 428a5997c83741a38eab94f3f245b137 P 3bdd3c39149842e5bd679e06e4609ce4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c46c18ff6384e02a65d5c945a90b15e in term 1.
I20260807 18:19:45.526254 13669 raft_consensus.cc:2454] T 428a5997c83741a38eab94f3f245b137 P e32f6f4c22594a90a800d40a6c9cfc8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c46c18ff6384e02a65d5c945a90b15e in term 1.
I20260807 18:19:45.526795 13771 leader_election.cc:302] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3bdd3c39149842e5bd679e06e4609ce4, 6c46c18ff6384e02a65d5c945a90b15e, e32f6f4c22594a90a800d40a6c9cfc8d; no voters:
I20260807 18:19:45.527150 14166 raft_consensus.cc:2781] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:19:45.527635 14166 raft_consensus.cc:686] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e [term 1 LEADER]: Becoming Leader. State: Replica: 6c46c18ff6384e02a65d5c945a90b15e, State: Running, Role: LEADER
I20260807 18:19:45.528301 14166 consensus_queue.cc:237] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c46c18ff6384e02a65d5c945a90b15e" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 39429 } } peers { permanent_uuid: "e32f6f4c22594a90a800d40a6c9cfc8d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 41945 } } peers { permanent_uuid: "3bdd3c39149842e5bd679e06e4609ce4" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 35821 } }
I20260807 18:19:45.533775 13439 catalog_manager.cc:5495] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c46c18ff6384e02a65d5c945a90b15e (127.17.110.2). New cstate: current_term: 1 leader_uuid: "6c46c18ff6384e02a65d5c945a90b15e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c46c18ff6384e02a65d5c945a90b15e" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 39429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e32f6f4c22594a90a800d40a6c9cfc8d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 41945 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bdd3c39149842e5bd679e06e4609ce4" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 35821 } health_report { overall_health: UNKNOWN } } }
I20260807 18:19:45.589963 14153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:45.584386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36924 (local address 127.17.110.2:39429)
0807 18:19:45.584610 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:45.584615 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:45.584636 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:19:45.584738 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:45.584742 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:45.584829 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:45.584994 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:19:45.585402 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.586257 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.587741 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.588191 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.588300 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:45.588763 (+ 463us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:19:45.589330 (+ 567us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:19:45.589445 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:45.589566 (+ 121us) server_negotiation.cc:299] Negotiation successful
0807 18:19:45.589640 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 18:19:45.599802 14060 raft_consensus.cc:1261] T 428a5997c83741a38eab94f3f245b137 P 3bdd3c39149842e5bd679e06e4609ce4 [term 1 FOLLOWER]: Refusing update from remote peer 6c46c18ff6384e02a65d5c945a90b15e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:19:45.600844 13669 raft_consensus.cc:1261] T 428a5997c83741a38eab94f3f245b137 P e32f6f4c22594a90a800d40a6c9cfc8d [term 1 FOLLOWER]: Refusing update from remote peer 6c46c18ff6384e02a65d5c945a90b15e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:19:45.601059 14172 consensus_queue.cc:1035] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bdd3c39149842e5bd679e06e4609ce4" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 35821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:19:45.602079 14166 consensus_queue.cc:1035] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e32f6f4c22594a90a800d40a6c9cfc8d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 41945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:19:45.617322 14177 mvcc.cc:205] Tried to move back new op lower bound from 7315975313803603968 to 7315975313528688640. Current Snapshot: MvccSnapshot[applied={T|T < 7315975313803603968}]
I20260807 18:19:45.618009 14176 mvcc.cc:205] Tried to move back new op lower bound from 7315975313803603968 to 7315975313528688640. Current Snapshot: MvccSnapshot[applied={T|T < 7315975313803603968}]
I20260807 18:19:45.636096 13439 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:19:45.636406 13439 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:19:45.640568 13439 catalog_manager.cc:5782] T 00000000000000000000000000000000 P af3ccefde21b4d9e8d589ec862ef04c2: Sending DeleteTablet for 3 replicas of tablet 428a5997c83741a38eab94f3f245b137
I20260807 18:19:45.642151 13833 tablet_service.cc:1514] Processing DeleteTablet for tablet 428a5997c83741a38eab94f3f245b137 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:19:45 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:36922
I20260807 18:19:45.642132 13638 tablet_service.cc:1514] Processing DeleteTablet for tablet 428a5997c83741a38eab94f3f245b137 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:19:45 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:58466
I20260807 18:19:45.642136 14033 tablet_service.cc:1514] Processing DeleteTablet for tablet 428a5997c83741a38eab94f3f245b137 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:19:45 UTC) from {username='kudu', principal='kudu/127.17.110.62@KRBTEST.COM'} at 127.0.0.1:44742
I20260807 18:19:45.646340 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 13530
I20260807 18:19:45.656512 14188 tablet_replica.cc:330] T 428a5997c83741a38eab94f3f245b137 P 3bdd3c39149842e5bd679e06e4609ce4: stopping tablet replica
I20260807 18:19:45.657927 14188 raft_consensus.cc:2229] T 428a5997c83741a38eab94f3f245b137 P 3bdd3c39149842e5bd679e06e4609ce4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:19:45.659413 14188 raft_consensus.cc:2258] T 428a5997c83741a38eab94f3f245b137 P 3bdd3c39149842e5bd679e06e4609ce4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:19:45.663249 14188 ts_tablet_manager.cc:1905] T 428a5997c83741a38eab94f3f245b137 P 3bdd3c39149842e5bd679e06e4609ce4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:19:45.663496 14187 tablet_replica.cc:330] T 428a5997c83741a38eab94f3f245b137 P e32f6f4c22594a90a800d40a6c9cfc8d: stopping tablet replica
I20260807 18:19:45.664125 14187 raft_consensus.cc:2229] T 428a5997c83741a38eab94f3f245b137 P e32f6f4c22594a90a800d40a6c9cfc8d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:19:45.664832 14187 raft_consensus.cc:2258] T 428a5997c83741a38eab94f3f245b137 P e32f6f4c22594a90a800d40a6c9cfc8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:19:45.666966 14185 tablet_replica.cc:330] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e: stopping tablet replica
I20260807 18:19:45.667268 14187 ts_tablet_manager.cc:1905] T 428a5997c83741a38eab94f3f245b137 P e32f6f4c22594a90a800d40a6c9cfc8d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:19:45.667524 14185 raft_consensus.cc:2229] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:19:45.668366 14185 raft_consensus.cc:2258] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 07 18:19:45 dist-test-slave-39l8 krb5kdc[13373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.1@KRBTEST.COM
I20260807 18:19:45.672019 14188 ts_tablet_manager.cc:1918] T 428a5997c83741a38eab94f3f245b137 P 3bdd3c39149842e5bd679e06e4609ce4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:19:45.672314 14188 log.cc:1196] T 428a5997c83741a38eab94f3f245b137 P 3bdd3c39149842e5bd679e06e4609ce4: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-2/wal/wals/428a5997c83741a38eab94f3f245b137
I20260807 18:19:45.672992 14188 ts_tablet_manager.cc:1939] T 428a5997c83741a38eab94f3f245b137 P 3bdd3c39149842e5bd679e06e4609ce4: Deleting consensus metadata
I20260807 18:19:45.675298 14187 ts_tablet_manager.cc:1918] T 428a5997c83741a38eab94f3f245b137 P e32f6f4c22594a90a800d40a6c9cfc8d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:19:45.675524 14187 log.cc:1196] T 428a5997c83741a38eab94f3f245b137 P e32f6f4c22594a90a800d40a6c9cfc8d: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-0/wal/wals/428a5997c83741a38eab94f3f245b137
I20260807 18:19:45.676106 14187 ts_tablet_manager.cc:1939] T 428a5997c83741a38eab94f3f245b137 P e32f6f4c22594a90a800d40a6c9cfc8d: Deleting consensus metadata
I20260807 18:19:45.676414 13423 catalog_manager.cc:4841] TS 3bdd3c39149842e5bd679e06e4609ce4 (127.17.110.3:35821): tablet 428a5997c83741a38eab94f3f245b137 (table test-table [id=48c65df43bf24fba9978b57882ca74ae]) successfully deleted
I20260807 18:19:45.677726 14185 ts_tablet_manager.cc:1905] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:19:45.678130 13418 catalog_manager.cc:4841] TS e32f6f4c22594a90a800d40a6c9cfc8d (127.17.110.1:41945): tablet 428a5997c83741a38eab94f3f245b137 (table test-table [id=48c65df43bf24fba9978b57882ca74ae]) successfully deleted
I20260807 18:19:45.681962 14151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:45.647649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58472 (local address 127.17.110.1:41945)
0807 18:19:45.647955 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:45.647964 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:45.653257 (+ 5293us) server_negotiation.cc:407] Connection header received
0807 18:19:45.654527 (+ 1270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:45.654537 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:45.654731 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:45.654860 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:45.657488 (+ 2628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.658856 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.661260 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.661965 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.662108 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:45.671627 (+ 9519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:45.671647 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:45.671666 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:19:45.671722 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:45.675976 (+ 4254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:45.679217 (+ 3241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:45.679227 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:45.679231 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:45.679411 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:45.679892 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:45.679899 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:45.679903 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:45.680401 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:45.680731 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:45.681067 (+ 336us) server_negotiation.cc:299] Negotiation successful
0807 18:19:45.681327 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260807 18:19:45.682689 13730 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:19:45.686252 14185 ts_tablet_manager.cc:1918] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:19:45.686450 14185 log.cc:1196] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126429568977-17848-0/minicluster-data/ts-1/wal/wals/428a5997c83741a38eab94f3f245b137
I20260807 18:19:45.687031 14185 ts_tablet_manager.cc:1939] T 428a5997c83741a38eab94f3f245b137 P 6c46c18ff6384e02a65d5c945a90b15e: Deleting consensus metadata
I20260807 18:19:45.688694 13423 catalog_manager.cc:4841] TS 6c46c18ff6384e02a65d5c945a90b15e (127.17.110.2:39429): tablet 428a5997c83741a38eab94f3f245b137 (table test-table [id=48c65df43bf24fba9978b57882ca74ae]) successfully deleted
I20260807 18:19:45.803627 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 13530
I20260807 18:19:45.827663 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 13744
Aug 07 18:19:45 dist-test-slave-39l8 krb5kdc[13373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.2@KRBTEST.COM
I20260807 18:19:45.845162 14153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:45.828495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36934 (local address 127.17.110.2:39429)
0807 18:19:45.828710 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:45.828719 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:45.828763 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:19:45.829019 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:45.829028 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:45.829098 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:45.829205 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:45.830558 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.831485 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.832931 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.833391 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.833537 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:45.838496 (+ 4959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:45.838513 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:45.838535 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:19:45.838588 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:45.841889 (+ 3301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:45.842853 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:45.842860 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:45.842862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:45.843005 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:45.843437 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:45.843440 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:45.843442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:45.843828 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:45.844206 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:45.844560 (+ 354us) server_negotiation.cc:299] Negotiation successful
0807 18:19:45.844794 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260807 18:19:45.845965 13916 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:19:45.964569 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 13744
I20260807 18:19:45.990027 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 13934
Aug 07 18:19:45 dist-test-slave-39l8 krb5kdc[13373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.3@KRBTEST.COM
I20260807 18:19:46.007226 14155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:45.990977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44746 (local address 127.17.110.3:35821)
0807 18:19:45.991160 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:45.991165 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:45.991186 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:19:45.991273 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:45.991278 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:45.991350 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:45.991452 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:45.992986 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.993820 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.995395 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:45.995800 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:45.995909 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:46.000893 (+ 4984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:46.000910 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:46.000928 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:19:46.000977 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:46.004021 (+ 3044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:46.004929 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:46.004937 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:46.004939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:46.005090 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:46.005483 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:46.005486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:46.005488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:46.005900 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:46.006127 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:46.006470 (+ 343us) server_negotiation.cc:299] Negotiation successful
0807 18:19:46.006651 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:19:46.007977 14127 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:19:46.106676 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 13934
I20260807 18:19:46.135813 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 13399
I20260807 18:19:46.137149 13486 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:19:46.251801 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 13399
2026-08-07T18:19:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (5167 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T18:19:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:19:46Z Disabled control of system clock
I20260807 18:19:46.357859 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:39597
--webserver_interface=127.17.110.62
--webserver_port=0
--builtin_ntp_servers=127.17.110.20:37199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:39597
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:19:46.730448 14199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:19:46.730849 14199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:19:46.731005 14199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:19:46.738644 14199 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:19:46.738868 14199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:19:46.738945 14199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:19:46.739043 14199 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:19:46.739113 14199 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:19:46.748826 14199 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:37199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:39597
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:39597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:19:46.750226 14199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:19:46.752229 14199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:19:46.764851 14211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:46.764895 14212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:46.764942 14216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:19:46.765966 14199 server_base.cc:1034] running on GCE node
I20260807 18:19:46.766929 14199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:19:46.768682 14199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:19:46.769990 14199 hybrid_clock.cc:648] HybridClock initialized: now 1786126786769931 us; error 43 us; skew 500 ppm
I20260807 18:19:46.770452 14199 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:19:46.773347 14199 webserver.cc:460] Webserver started at http://127.17.110.62:39895/ using document root <none> and password file <none>
I20260807 18:19:46.774187 14199 fs_manager.cc:359] Metadata directory not provided
I20260807 18:19:46.774294 14199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:19:46.774688 14199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:19:46.778119 14199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "a2a7d4ded6b44afe97b3aee792e61994"
format_stamp: "Formatted at 2026-08-07 18:19:46 on dist-test-slave-39l8"
server_key: "a194b9d458c17ddcb9ac82370890bc79"
server_key_iv: "47b7c9e17d67c3ca2cee6e02cc88ee6e"
server_key_version: "encryptionkey@0"
I20260807 18:19:46.779203 14199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "a2a7d4ded6b44afe97b3aee792e61994"
format_stamp: "Formatted at 2026-08-07 18:19:46 on dist-test-slave-39l8"
server_key: "a194b9d458c17ddcb9ac82370890bc79"
server_key_iv: "47b7c9e17d67c3ca2cee6e02cc88ee6e"
server_key_version: "encryptionkey@0"
I20260807 18:19:46.785583 14199 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260807 18:19:46.789902 14225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:46.791798 14199 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:19:46.792096 14199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "a2a7d4ded6b44afe97b3aee792e61994"
format_stamp: "Formatted at 2026-08-07 18:19:46 on dist-test-slave-39l8"
server_key: "a194b9d458c17ddcb9ac82370890bc79"
server_key_iv: "47b7c9e17d67c3ca2cee6e02cc88ee6e"
server_key_version: "encryptionkey@0"
I20260807 18:19:46.792382 14199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:19:46.819535 14199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:19:46.820531 14199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:19:46.820876 14199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:19:46.842072 14199 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:39597
I20260807 18:19:46.842128 14292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:39597 every 8 connection(s)
I20260807 18:19:46.843688 14199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:19:46.847774 14293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:19:46.852706 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 14199
I20260807 18:19:46.852909 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:19:46.853404 17848 external_mini_cluster.cc:1428] Setting key 8bbe93fe72eb57f69386a81d22ba9653
I20260807 18:19:46.860775 14293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2a7d4ded6b44afe97b3aee792e61994: Bootstrap starting.
I20260807 18:19:46.864629 14293 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2a7d4ded6b44afe97b3aee792e61994: Neither blocks nor log segments found. Creating new log.
I20260807 18:19:46.865840 14293 log.cc:824] T 00000000000000000000000000000000 P a2a7d4ded6b44afe97b3aee792e61994: Log is configured to *not* fsync() on all Append() calls
I20260807 18:19:46.870121 14293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2a7d4ded6b44afe97b3aee792e61994: No bootstrap required, opened a new log
I20260807 18:19:46.870862 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:46.855655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54276 (local address 127.17.110.62:39597)
0807 18:19:46.856872 (+ 1217us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:46.856893 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:46.856951 (+ 58us) server_negotiation.cc:407] Connection header received
0807 18:19:46.859530 (+ 2579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:46.859562 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:46.860192 (+ 630us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:46.860674 (+ 482us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:46.861262 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:46.863347 (+ 2085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:46.864903 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:46.865731 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:46.865984 (+ 253us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:46.867438 (+ 1454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:46.867447 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:46.867504 (+ 57us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:19:46.868348 (+ 844us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:46.868469 (+ 121us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:19:46.868501 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:46.868683 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:46.868919 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 18:19:46.869321 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":376,"threads_started":1}
I20260807 18:19:46.878938 14293 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2a7d4ded6b44afe97b3aee792e61994 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a7d4ded6b44afe97b3aee792e61994" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 39597 } }
I20260807 18:19:46.879397 14293 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2a7d4ded6b44afe97b3aee792e61994 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:19:46.879528 14293 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2a7d4ded6b44afe97b3aee792e61994 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2a7d4ded6b44afe97b3aee792e61994, State: Initialized, Role: FOLLOWER
I20260807 18:19:46.880192 14293 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2a7d4ded6b44afe97b3aee792e61994 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a7d4ded6b44afe97b3aee792e61994" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 39597 } }
I20260807 18:19:46.880467 14293 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2a7d4ded6b44afe97b3aee792e61994 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:19:46.880671 14293 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2a7d4ded6b44afe97b3aee792e61994 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:19:46.880856 14293 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2a7d4ded6b44afe97b3aee792e61994 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:19:46.883448 14293 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2a7d4ded6b44afe97b3aee792e61994 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a7d4ded6b44afe97b3aee792e61994" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 39597 } }
I20260807 18:19:46.884019 14293 leader_election.cc:302] T 00000000000000000000000000000000 P a2a7d4ded6b44afe97b3aee792e61994 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2a7d4ded6b44afe97b3aee792e61994; no voters:
I20260807 18:19:46.884677 14293 leader_election.cc:288] T 00000000000000000000000000000000 P a2a7d4ded6b44afe97b3aee792e61994 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:19:46.884865 14300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2a7d4ded6b44afe97b3aee792e61994 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:19:46.885644 14300 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2a7d4ded6b44afe97b3aee792e61994 [term 1 LEADER]: Becoming Leader. State: Replica: a2a7d4ded6b44afe97b3aee792e61994, State: Running, Role: LEADER
I20260807 18:19:46.886438 14300 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2a7d4ded6b44afe97b3aee792e61994 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a7d4ded6b44afe97b3aee792e61994" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 39597 } }
I20260807 18:19:46.887023 14293 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2a7d4ded6b44afe97b3aee792e61994 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:19:46.889597 14302 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2a7d4ded6b44afe97b3aee792e61994 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2a7d4ded6b44afe97b3aee792e61994" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a7d4ded6b44afe97b3aee792e61994" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 39597 } } }
I20260807 18:19:46.890285 14302 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2a7d4ded6b44afe97b3aee792e61994 [sys.catalog]: This master's current role is: LEADER
I20260807 18:19:46.889698 14303 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2a7d4ded6b44afe97b3aee792e61994 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2a7d4ded6b44afe97b3aee792e61994. Latest consensus state: current_term: 1 leader_uuid: "a2a7d4ded6b44afe97b3aee792e61994" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a7d4ded6b44afe97b3aee792e61994" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 39597 } } }
I20260807 18:19:46.892586 14303 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2a7d4ded6b44afe97b3aee792e61994 [sys.catalog]: This master's current role is: LEADER
I20260807 18:19:46.892766 14310 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:19:46.898886 14310 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:19:46.909729 14310 catalog_manager.cc:1347] Generated new cluster ID: 16f15c59f15d418290592910078a47e1
I20260807 18:19:46.909881 14310 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:19:46.919759 14310 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:19:46.921185 14310 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:19:46.931917 14310 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2a7d4ded6b44afe97b3aee792e61994: Generated new TSK 0
I20260807 18:19:46.933184 14310 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:19:46.947075 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.1:0
--local_ip_for_outbound_sockets=127.17.110.1
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:39597
--builtin_ntp_servers=127.17.110.20:37199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:19:47.314647 14335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:19:47.315161 14335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:19:47.315313 14335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:19:47.323148 14335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:19:47.323303 14335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:19:47.323475 14335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.1
I20260807 18:19:47.333199 14335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:37199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.110.1
--webserver_port=0
--tserver_master_addrs=127.17.110.62:39597
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.1
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:19:47.334612 14335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:19:47.336560 14335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:19:47.350984 14346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:47.351042 14344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:47.351070 14343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:19:47.351938 14335 server_base.cc:1034] running on GCE node
I20260807 18:19:47.352684 14335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:19:47.353772 14335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:19:47.355021 14335 hybrid_clock.cc:648] HybridClock initialized: now 1786126787354965 us; error 47 us; skew 500 ppm
I20260807 18:19:47.355435 14335 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:19:47.357976 14335 webserver.cc:460] Webserver started at http://127.17.110.1:35817/ using document root <none> and password file <none>
I20260807 18:19:47.358914 14335 fs_manager.cc:359] Metadata directory not provided
I20260807 18:19:47.359058 14335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:19:47.359417 14335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:19:47.362598 14335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-0/data/instance:
uuid: "5551a0d5d1ec44b9873902f083d1a00d"
format_stamp: "Formatted at 2026-08-07 18:19:47 on dist-test-slave-39l8"
server_key: "4ff7660802de3bbb507594024f159ad2"
server_key_iv: "c75d2b2b069ebd30eecc65dbed56303c"
server_key_version: "encryptionkey@0"
I20260807 18:19:47.363579 14335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance:
uuid: "5551a0d5d1ec44b9873902f083d1a00d"
format_stamp: "Formatted at 2026-08-07 18:19:47 on dist-test-slave-39l8"
server_key: "4ff7660802de3bbb507594024f159ad2"
server_key_iv: "c75d2b2b069ebd30eecc65dbed56303c"
server_key_version: "encryptionkey@0"
I20260807 18:19:47.369628 14335 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.004s
I20260807 18:19:47.373862 14352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:47.375628 14335 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:19:47.375861 14335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-0/wal
uuid: "5551a0d5d1ec44b9873902f083d1a00d"
format_stamp: "Formatted at 2026-08-07 18:19:47 on dist-test-slave-39l8"
server_key: "4ff7660802de3bbb507594024f159ad2"
server_key_iv: "c75d2b2b069ebd30eecc65dbed56303c"
server_key_version: "encryptionkey@0"
I20260807 18:19:47.376212 14335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:19:47.405917 14335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:19:47.406972 14335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:19:47.407323 14335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:19:47.408401 14335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:19:47.410017 14335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:19:47.410159 14335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:47.410290 14335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:19:47.410382 14335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:47.447345 14335 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.1:40505
I20260807 18:19:47.447438 14527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.1:40505 every 8 connection(s)
I20260807 18:19:47.448736 14335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-0/data/info.pb
I20260807 18:19:47.455076 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 14335
I20260807 18:19:47.455276 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-0/wal/instance
I20260807 18:19:47.455767 17848 external_mini_cluster.cc:1428] Setting key 65dd4c2228f411917a5fbe28653fb0f8
I20260807 18:19:47.458674 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.2:0
--local_ip_for_outbound_sockets=127.17.110.2
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:39597
--builtin_ntp_servers=127.17.110.20:37199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:19:47.462371 14530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:47.451656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.1:52421 (local address 127.17.110.62:39597)
0807 18:19:47.452171 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:47.452176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:47.453335 (+ 1159us) server_negotiation.cc:407] Connection header received
0807 18:19:47.455343 (+ 2008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:47.455349 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:47.455443 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:47.455565 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:47.456789 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:47.457605 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:47.459354 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:47.459788 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:47.459941 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:47.461470 (+ 1529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:47.461475 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:47.461478 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:19:47.461498 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:47.461539 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:19:47.461551 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:47.461657 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:47.461901 (+ 244us) server_negotiation.cc:299] Negotiation successful
0807 18:19:47.462014 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":279,"threads_started":1}
I20260807 18:19:47.463557 14531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:47.452193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:39597 (local address 127.17.110.1:52421)
0807 18:19:47.453109 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:47.453206 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:47.455012 (+ 1806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:47.455810 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:47.455875 (+ 65us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:47.456004 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:19:47.456599 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:47.456617 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:47.457775 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:47.457781 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:47.459199 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:47.459213 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:47.460047 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:47.460055 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:47.460303 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:47.461070 (+ 767us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:19:47.461073 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:19:47.461115 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:19:47.461260 (+ 145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:19:47.461656 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:19:47.461672 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:19:47.461683 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 18:19:47.461890 (+ 207us) client_negotiation.cc:240] Negotiation successful
0807 18:19:47.462344 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":269,"threads_started":1}
I20260807 18:19:47.465309 14528 heartbeater.cc:347] Connected to a master server at 127.17.110.62:39597
I20260807 18:19:47.465693 14528 heartbeater.cc:464] Registering TS with master...
I20260807 18:19:47.466784 14528 heartbeater.cc:511] Master 127.17.110.62:39597 requested a full tablet report, sending...
I20260807 18:19:47.469446 14246 ts_manager.cc:194] Registered new tserver with Master: 5551a0d5d1ec44b9873902f083d1a00d (127.17.110.1:40505)
I20260807 18:19:47.471946 14246 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.110.1:52421
W20260807 18:19:47.821086 14532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:19:47.821411 14532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:19:47.821560 14532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:19:47.829232 14532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:19:47.829356 14532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:19:47.829532 14532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.2
I20260807 18:19:47.839864 14532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:37199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.110.2
--webserver_port=0
--tserver_master_addrs=127.17.110.62:39597
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.2
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:19:47.841373 14532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:19:47.843864 14532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:19:47.861383 14543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:47.861517 14542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:47.861615 14545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:19:47.863519 14532 server_base.cc:1034] running on GCE node
I20260807 18:19:47.864329 14532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:19:47.865593 14532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:19:47.866868 14532 hybrid_clock.cc:648] HybridClock initialized: now 1786126787866812 us; error 48 us; skew 500 ppm
I20260807 18:19:47.867273 14532 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:19:47.869876 14532 webserver.cc:460] Webserver started at http://127.17.110.2:38953/ using document root <none> and password file <none>
I20260807 18:19:47.870697 14532 fs_manager.cc:359] Metadata directory not provided
I20260807 18:19:47.870901 14532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:19:47.871259 14532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:19:47.874579 14532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-1/data/instance:
uuid: "f52ac5e88a72496282c89a3654ae556f"
format_stamp: "Formatted at 2026-08-07 18:19:47 on dist-test-slave-39l8"
server_key: "214b067052367906f104d4272b6d2af1"
server_key_iv: "638e4e93a92826b0b80c4d06ae85bc10"
server_key_version: "encryptionkey@0"
I20260807 18:19:47.875517 14532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance:
uuid: "f52ac5e88a72496282c89a3654ae556f"
format_stamp: "Formatted at 2026-08-07 18:19:47 on dist-test-slave-39l8"
server_key: "214b067052367906f104d4272b6d2af1"
server_key_iv: "638e4e93a92826b0b80c4d06ae85bc10"
server_key_version: "encryptionkey@0"
I20260807 18:19:47.881681 14532 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 18:19:47.885650 14553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:47.887321 14532 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 18:19:47.887528 14532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-1/wal
uuid: "f52ac5e88a72496282c89a3654ae556f"
format_stamp: "Formatted at 2026-08-07 18:19:47 on dist-test-slave-39l8"
server_key: "214b067052367906f104d4272b6d2af1"
server_key_iv: "638e4e93a92826b0b80c4d06ae85bc10"
server_key_version: "encryptionkey@0"
I20260807 18:19:47.887791 14532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:19:47.905037 14532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:19:47.905953 14532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:19:47.906237 14532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:19:47.907271 14532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:19:47.908783 14532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:19:47.908876 14532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:47.908968 14532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:19:47.909026 14532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:47.943707 14532 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.2:41829
I20260807 18:19:47.943763 14710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.2:41829 every 8 connection(s)
I20260807 18:19:47.945106 14532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-1/data/info.pb
I20260807 18:19:47.949182 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 14532
I20260807 18:19:47.949419 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-1/wal/instance
I20260807 18:19:47.950049 17848 external_mini_cluster.cc:1428] Setting key 0b612c5a781c532cdb2efe0d014700db
I20260807 18:19:47.953686 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.110.3:0
--local_ip_for_outbound_sockets=127.17.110.3
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:39597
--builtin_ntp_servers=127.17.110.20:37199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:19:47.962003 14530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:47.948468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:52851 (local address 127.17.110.62:39597)
0807 18:19:47.948733 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:47.948740 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:47.950625 (+ 1885us) server_negotiation.cc:407] Connection header received
0807 18:19:47.953126 (+ 2501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:47.953130 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:47.953215 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:47.953305 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:47.954986 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:47.955760 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:47.958585 (+ 2825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:47.959113 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:47.959227 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:47.961126 (+ 1899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:47.961133 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:47.961138 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:19:47.961165 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:47.961215 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:19:47.961230 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:47.961317 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:47.961644 (+ 327us) server_negotiation.cc:299] Negotiation successful
0807 18:19:47.961738 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 18:19:47.963795 14714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:47.949098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:39597 (local address 127.17.110.2:52851)
0807 18:19:47.950359 (+ 1261us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:47.950414 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:47.952794 (+ 2380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:47.953639 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:47.953652 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:47.953797 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:19:47.954785 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:47.954818 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:47.955930 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:47.955939 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:47.958371 (+ 2432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:47.958392 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:47.959379 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:47.959390 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:47.959776 (+ 386us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:47.960678 (+ 902us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:19:47.960684 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:19:47.960748 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:19:47.960871 (+ 123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:19:47.961343 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:19:47.961359 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:19:47.961371 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 18:19:47.961604 (+ 233us) client_negotiation.cc:240] Negotiation successful
0807 18:19:47.962112 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":392,"threads_started":1}
I20260807 18:19:47.965445 14712 heartbeater.cc:347] Connected to a master server at 127.17.110.62:39597
I20260807 18:19:47.965826 14712 heartbeater.cc:464] Registering TS with master...
I20260807 18:19:47.966959 14712 heartbeater.cc:511] Master 127.17.110.62:39597 requested a full tablet report, sending...
I20260807 18:19:47.968859 14246 ts_manager.cc:194] Registered new tserver with Master: f52ac5e88a72496282c89a3654ae556f (127.17.110.2:41829)
I20260807 18:19:47.969856 14246 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.110.2:52851
W20260807 18:19:48.329202 14715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:19:48.329509 14715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:19:48.329664 14715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:19:48.337256 14715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:19:48.337393 14715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:19:48.337574 14715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.110.3
I20260807 18:19:48.347265 14715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:37199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.110.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.110.3
--webserver_port=0
--tserver_master_addrs=127.17.110.62:39597
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.110.3
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:19:48.348793 14715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:19:48.350641 14715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:19:48.364500 14721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:48.364532 14722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:48.364460 14726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:19:48.366317 14715 server_base.cc:1034] running on GCE node
I20260807 18:19:48.367123 14715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:19:48.368346 14715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:19:48.369601 14715 hybrid_clock.cc:648] HybridClock initialized: now 1786126788369530 us; error 59 us; skew 500 ppm
I20260807 18:19:48.370004 14715 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:19:48.375969 14715 webserver.cc:460] Webserver started at http://127.17.110.3:39075/ using document root <none> and password file <none>
I20260807 18:19:48.376856 14715 fs_manager.cc:359] Metadata directory not provided
I20260807 18:19:48.376989 14715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:19:48.377348 14715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:19:48.380759 14715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-2/data/instance:
uuid: "4eb90f2e49b74a9aaa6d5575fb8a8b08"
format_stamp: "Formatted at 2026-08-07 18:19:48 on dist-test-slave-39l8"
server_key: "fb7f1ae6f0ff7286b65656dd39effb9d"
server_key_iv: "8dbea679fc56e76d660fdc39a810d3eb"
server_key_version: "encryptionkey@0"
I20260807 18:19:48.381771 14715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance:
uuid: "4eb90f2e49b74a9aaa6d5575fb8a8b08"
format_stamp: "Formatted at 2026-08-07 18:19:48 on dist-test-slave-39l8"
server_key: "fb7f1ae6f0ff7286b65656dd39effb9d"
server_key_iv: "8dbea679fc56e76d660fdc39a810d3eb"
server_key_version: "encryptionkey@0"
I20260807 18:19:48.387724 14715 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 18:19:48.391974 14737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:48.393685 14715 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:19:48.393872 14715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-2/wal
uuid: "4eb90f2e49b74a9aaa6d5575fb8a8b08"
format_stamp: "Formatted at 2026-08-07 18:19:48 on dist-test-slave-39l8"
server_key: "fb7f1ae6f0ff7286b65656dd39effb9d"
server_key_iv: "8dbea679fc56e76d660fdc39a810d3eb"
server_key_version: "encryptionkey@0"
I20260807 18:19:48.394101 14715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:19:48.419430 14715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:19:48.420372 14715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:19:48.420703 14715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:19:48.421823 14715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:19:48.423530 14715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:19:48.423624 14715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:48.423804 14715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:19:48.423892 14715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:48.464295 14715 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.3:42259
I20260807 18:19:48.464413 14900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.3:42259 every 8 connection(s)
I20260807 18:19:48.465685 14715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-2/data/info.pb
I20260807 18:19:48.471596 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 14715
I20260807 18:19:48.471805 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-2/wal/instance
I20260807 18:19:48.472281 17848 external_mini_cluster.cc:1428] Setting key d15530ccdad558ac9c7c7cf713c5d1b7
I20260807 18:19:48.475798 14528 heartbeater.cc:503] Master 127.17.110.62:39597 was elected leader, sending a full tablet report...
I20260807 18:19:48.479941 14903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:48.468535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.3:43391 (local address 127.17.110.62:39597)
0807 18:19:48.469009 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:48.469014 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:48.470243 (+ 1229us) server_negotiation.cc:407] Connection header received
0807 18:19:48.472392 (+ 2149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:48.472398 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:48.472500 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:48.472624 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:48.473996 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:48.474865 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:48.476836 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:48.477263 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:48.477407 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:48.478998 (+ 1591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:48.479009 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:48.479012 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:19:48.479032 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:48.479074 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:19:48.479086 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:48.479186 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:48.479435 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 18:19:48.479544 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":242,"threads_started":1}
I20260807 18:19:48.481019 14904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:48.469110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:39597 (local address 127.17.110.3:43391)
0807 18:19:48.470049 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:48.470099 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:48.472064 (+ 1965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:48.472880 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:48.472908 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:48.473048 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:19:48.473816 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:48.473837 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:48.475038 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:48.475042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:48.476670 (+ 1628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:48.476686 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:48.477523 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:48.477532 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:48.477813 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:48.478583 (+ 770us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:19:48.478585 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:19:48.478630 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:19:48.478782 (+ 152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:19:48.479187 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:19:48.479201 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:19:48.479212 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 18:19:48.479420 (+ 208us) client_negotiation.cc:240] Negotiation successful
0807 18:19:48.479877 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":272,"threads_started":1}
I20260807 18:19:48.482833 14901 heartbeater.cc:347] Connected to a master server at 127.17.110.62:39597
I20260807 18:19:48.483281 14901 heartbeater.cc:464] Registering TS with master...
I20260807 18:19:48.484160 14901 heartbeater.cc:511] Master 127.17.110.62:39597 requested a full tablet report, sending...
I20260807 18:19:48.485832 14246 ts_manager.cc:194] Registered new tserver with Master: 4eb90f2e49b74a9aaa6d5575fb8a8b08 (127.17.110.3:42259)
I20260807 18:19:48.487056 14246 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.110.3:43391
I20260807 18:19:48.488656 17848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:19:48.500931 14903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:48.494102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54280 (local address 127.17.110.62:39597)
0807 18:19:48.494333 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:48.494340 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:48.494877 (+ 537us) server_negotiation.cc:407] Connection header received
0807 18:19:48.495126 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:48.495131 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:48.495210 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:48.495316 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:48.495916 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:48.496782 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:48.498250 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:48.498830 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:48.498976 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:48.500128 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:48.500133 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:48.500136 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:19:48.500159 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:48.500196 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:19:48.500208 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:48.500308 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:48.500457 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 18:19:48.500601 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 18:19:48.505877 14246 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54280:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:19:48.513185 14246 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:19:48.551796 14927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:48.540128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55978 (local address 127.17.110.3:42259)
0807 18:19:48.544036 (+ 3908us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:48.544047 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:48.544597 (+ 550us) server_negotiation.cc:407] Connection header received
0807 18:19:48.544800 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:48.544806 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:48.544996 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:48.545214 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:19:48.545870 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:48.547040 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:48.549465 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:48.550643 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:48.550837 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:48.551111 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:48.551224 (+ 113us) server_negotiation.cc:299] Negotiation successful
0807 18:19:48.551407 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3693,"thread_start_us":3520,"threads_started":1}
I20260807 18:19:48.551795 14928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:48.540670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:42259 (local address 127.0.0.1:55978)
0807 18:19:48.544463 (+ 3793us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:48.544484 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:48.544651 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:48.545253 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:48.545261 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:48.545292 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:19:48.545694 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:48.545704 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:48.547220 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:48.547226 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:48.549266 (+ 2040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:48.549279 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:48.550874 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:48.550880 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:48.551036 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:48.551077 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 18:19:48.551150 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 18:19:48.551367 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3637,"mutex_wait_us":104,"thread_start_us":3413,"threads_started":1}
I20260807 18:19:48.552898 14926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:48.527946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38382 (local address 127.17.110.1:40505)
0807 18:19:48.538618 (+ 10672us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:48.538632 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:48.539639 (+ 1007us) server_negotiation.cc:407] Connection header received
0807 18:19:48.539792 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:48.539804 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:48.540173 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:48.541356 (+ 1183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:19:48.541681 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:48.543783 (+ 2102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:48.548292 (+ 4509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:48.549932 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:48.550082 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:48.550337 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:48.552322 (+ 1985us) server_negotiation.cc:299] Negotiation successful
0807 18:19:48.552526 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10394,"thread_start_us":10217,"threads_started":1}
I20260807 18:19:48.553290 14925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:48.526613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.2:41829 (local address 127.0.0.1:48614)
0807 18:19:48.538781 (+ 12168us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:48.538864 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:48.539142 (+ 278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:48.541816 (+ 2674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:48.541825 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:48.541868 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:19:48.545017 (+ 3149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:48.545032 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:48.547161 (+ 2129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:48.547167 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:48.549822 (+ 2655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:48.549839 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:48.552089 (+ 2250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:48.552098 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:48.552304 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:48.552339 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 18:19:48.552432 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 18:19:48.552561 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11731,"mutex_wait_us":174,"thread_start_us":12150,"threads_started":1}
I20260807 18:19:48.553330 14922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:48.527566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:40505 (local address 127.0.0.1:38382)
0807 18:19:48.539203 (+ 11637us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:48.539232 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:48.539436 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:48.540490 (+ 1054us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:48.540501 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:48.540538 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:19:48.541055 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:48.541076 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:48.543994 (+ 2918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:48.544000 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:48.547892 (+ 3892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:48.547910 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:48.550446 (+ 2536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:48.550451 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:48.550590 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:48.550625 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 18:19:48.550812 (+ 187us) client_negotiation.cc:240] Negotiation successful
0807 18:19:48.552727 (+ 1915us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11427,"mutex_wait_us":4,"thread_start_us":11750,"threads_started":1}
I20260807 18:19:48.553676 14923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:48.526893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48614 (local address 127.17.110.2:41829)
0807 18:19:48.540606 (+ 13713us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:48.540636 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:48.540679 (+ 43us) server_negotiation.cc:407] Connection header received
0807 18:19:48.540829 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:48.540842 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:48.541164 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:48.541466 (+ 302us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:19:48.545285 (+ 3819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:48.546980 (+ 1695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:48.550054 (+ 3074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:48.551527 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:48.551678 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:48.551929 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:48.553095 (+ 1166us) server_negotiation.cc:299] Negotiation successful
0807 18:19:48.553285 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13485,"thread_start_us":13362,"threads_started":1}
I20260807 18:19:48.556099 14806 tablet_service.cc:1467] Processing CreateTablet for tablet 88bcba26644e4ec997ba62b58db3d68c (DEFAULT_TABLE table=test-table [id=0e2694e20c6045c4807d476f3baafc92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:19:48.557245 14618 tablet_service.cc:1467] Processing CreateTablet for tablet 88bcba26644e4ec997ba62b58db3d68c (DEFAULT_TABLE table=test-table [id=0e2694e20c6045c4807d476f3baafc92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:19:48.557245 14435 tablet_service.cc:1467] Processing CreateTablet for tablet 88bcba26644e4ec997ba62b58db3d68c (DEFAULT_TABLE table=test-table [id=0e2694e20c6045c4807d476f3baafc92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:19:48.558395 14806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88bcba26644e4ec997ba62b58db3d68c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:19:48.558686 14435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88bcba26644e4ec997ba62b58db3d68c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:19:48.558692 14618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88bcba26644e4ec997ba62b58db3d68c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:19:48.573498 14929 tablet_bootstrap.cc:492] T 88bcba26644e4ec997ba62b58db3d68c P 4eb90f2e49b74a9aaa6d5575fb8a8b08: Bootstrap starting.
I20260807 18:19:48.575584 14931 tablet_bootstrap.cc:492] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f: Bootstrap starting.
I20260807 18:19:48.577075 14932 tablet_bootstrap.cc:492] T 88bcba26644e4ec997ba62b58db3d68c P 5551a0d5d1ec44b9873902f083d1a00d: Bootstrap starting.
I20260807 18:19:48.578408 14929 tablet_bootstrap.cc:654] T 88bcba26644e4ec997ba62b58db3d68c P 4eb90f2e49b74a9aaa6d5575fb8a8b08: Neither blocks nor log segments found. Creating new log.
I20260807 18:19:48.579977 14929 log.cc:824] T 88bcba26644e4ec997ba62b58db3d68c P 4eb90f2e49b74a9aaa6d5575fb8a8b08: Log is configured to *not* fsync() on all Append() calls
I20260807 18:19:48.581297 14932 tablet_bootstrap.cc:654] T 88bcba26644e4ec997ba62b58db3d68c P 5551a0d5d1ec44b9873902f083d1a00d: Neither blocks nor log segments found. Creating new log.
I20260807 18:19:48.582751 14931 tablet_bootstrap.cc:654] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f: Neither blocks nor log segments found. Creating new log.
I20260807 18:19:48.582996 14932 log.cc:824] T 88bcba26644e4ec997ba62b58db3d68c P 5551a0d5d1ec44b9873902f083d1a00d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:19:48.584057 14931 log.cc:824] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f: Log is configured to *not* fsync() on all Append() calls
I20260807 18:19:48.586663 14932 tablet_bootstrap.cc:492] T 88bcba26644e4ec997ba62b58db3d68c P 5551a0d5d1ec44b9873902f083d1a00d: No bootstrap required, opened a new log
I20260807 18:19:48.587038 14932 ts_tablet_manager.cc:1397] T 88bcba26644e4ec997ba62b58db3d68c P 5551a0d5d1ec44b9873902f083d1a00d: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260807 18:19:48.587528 14929 tablet_bootstrap.cc:492] T 88bcba26644e4ec997ba62b58db3d68c P 4eb90f2e49b74a9aaa6d5575fb8a8b08: No bootstrap required, opened a new log
I20260807 18:19:48.587581 14931 tablet_bootstrap.cc:492] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f: No bootstrap required, opened a new log
I20260807 18:19:48.587867 14929 ts_tablet_manager.cc:1397] T 88bcba26644e4ec997ba62b58db3d68c P 4eb90f2e49b74a9aaa6d5575fb8a8b08: Time spent bootstrapping tablet: real 0.015s user 0.011s sys 0.000s
I20260807 18:19:48.587862 14931 ts_tablet_manager.cc:1397] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.000s
I20260807 18:19:48.596561 14932 raft_consensus.cc:348] T 88bcba26644e4ec997ba62b58db3d68c P 5551a0d5d1ec44b9873902f083d1a00d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f52ac5e88a72496282c89a3654ae556f" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 41829 } } peers { permanent_uuid: "4eb90f2e49b74a9aaa6d5575fb8a8b08" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 42259 } } peers { permanent_uuid: "5551a0d5d1ec44b9873902f083d1a00d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 40505 } }
I20260807 18:19:48.597110 14932 raft_consensus.cc:374] T 88bcba26644e4ec997ba62b58db3d68c P 5551a0d5d1ec44b9873902f083d1a00d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:19:48.597226 14932 raft_consensus.cc:729] T 88bcba26644e4ec997ba62b58db3d68c P 5551a0d5d1ec44b9873902f083d1a00d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5551a0d5d1ec44b9873902f083d1a00d, State: Initialized, Role: FOLLOWER
I20260807 18:19:48.597996 14932 consensus_queue.cc:260] T 88bcba26644e4ec997ba62b58db3d68c P 5551a0d5d1ec44b9873902f083d1a00d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f52ac5e88a72496282c89a3654ae556f" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 41829 } } peers { permanent_uuid: "4eb90f2e49b74a9aaa6d5575fb8a8b08" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 42259 } } peers { permanent_uuid: "5551a0d5d1ec44b9873902f083d1a00d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 40505 } }
I20260807 18:19:48.600836 14931 raft_consensus.cc:348] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f52ac5e88a72496282c89a3654ae556f" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 41829 } } peers { permanent_uuid: "4eb90f2e49b74a9aaa6d5575fb8a8b08" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 42259 } } peers { permanent_uuid: "5551a0d5d1ec44b9873902f083d1a00d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 40505 } }
I20260807 18:19:48.600836 14929 raft_consensus.cc:348] T 88bcba26644e4ec997ba62b58db3d68c P 4eb90f2e49b74a9aaa6d5575fb8a8b08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f52ac5e88a72496282c89a3654ae556f" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 41829 } } peers { permanent_uuid: "4eb90f2e49b74a9aaa6d5575fb8a8b08" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 42259 } } peers { permanent_uuid: "5551a0d5d1ec44b9873902f083d1a00d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 40505 } }
I20260807 18:19:48.601385 14932 ts_tablet_manager.cc:1428] T 88bcba26644e4ec997ba62b58db3d68c P 5551a0d5d1ec44b9873902f083d1a00d: Time spent starting tablet: real 0.014s user 0.005s sys 0.007s
I20260807 18:19:48.601393 14929 raft_consensus.cc:374] T 88bcba26644e4ec997ba62b58db3d68c P 4eb90f2e49b74a9aaa6d5575fb8a8b08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:19:48.601390 14931 raft_consensus.cc:374] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:19:48.601507 14931 raft_consensus.cc:729] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f52ac5e88a72496282c89a3654ae556f, State: Initialized, Role: FOLLOWER
I20260807 18:19:48.601507 14929 raft_consensus.cc:729] T 88bcba26644e4ec997ba62b58db3d68c P 4eb90f2e49b74a9aaa6d5575fb8a8b08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4eb90f2e49b74a9aaa6d5575fb8a8b08, State: Initialized, Role: FOLLOWER
I20260807 18:19:48.602274 14929 consensus_queue.cc:260] T 88bcba26644e4ec997ba62b58db3d68c P 4eb90f2e49b74a9aaa6d5575fb8a8b08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f52ac5e88a72496282c89a3654ae556f" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 41829 } } peers { permanent_uuid: "4eb90f2e49b74a9aaa6d5575fb8a8b08" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 42259 } } peers { permanent_uuid: "5551a0d5d1ec44b9873902f083d1a00d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 40505 } }
I20260807 18:19:48.602274 14931 consensus_queue.cc:260] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f52ac5e88a72496282c89a3654ae556f" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 41829 } } peers { permanent_uuid: "4eb90f2e49b74a9aaa6d5575fb8a8b08" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 42259 } } peers { permanent_uuid: "5551a0d5d1ec44b9873902f083d1a00d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 40505 } }
I20260807 18:19:48.604107 14712 heartbeater.cc:503] Master 127.17.110.62:39597 was elected leader, sending a full tablet report...
I20260807 18:19:48.604516 14929 ts_tablet_manager.cc:1428] T 88bcba26644e4ec997ba62b58db3d68c P 4eb90f2e49b74a9aaa6d5575fb8a8b08: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
I20260807 18:19:48.604794 14931 ts_tablet_manager.cc:1428] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f: Time spent starting tablet: real 0.017s user 0.012s sys 0.005s
I20260807 18:19:48.606091 14901 heartbeater.cc:503] Master 127.17.110.62:39597 was elected leader, sending a full tablet report...
I20260807 18:19:48.693506 14944 raft_consensus.cc:482] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:19:48.693925 14944 raft_consensus.cc:504] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f52ac5e88a72496282c89a3654ae556f" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 41829 } } peers { permanent_uuid: "4eb90f2e49b74a9aaa6d5575fb8a8b08" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 42259 } } peers { permanent_uuid: "5551a0d5d1ec44b9873902f083d1a00d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 40505 } }
I20260807 18:19:48.695761 14944 leader_election.cc:288] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4eb90f2e49b74a9aaa6d5575fb8a8b08 (127.17.110.3:42259), 5551a0d5d1ec44b9873902f083d1a00d (127.17.110.1:40505)
W20260807 18:19:48.697476 14713 tablet.cc:2369] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:19:48.701447 14529 tablet.cc:2369] T 88bcba26644e4ec997ba62b58db3d68c P 5551a0d5d1ec44b9873902f083d1a00d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:19:48.704818 14927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:48.696356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:58051 (local address 127.17.110.3:42259)
0807 18:19:48.696581 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:48.696589 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:48.698106 (+ 1517us) server_negotiation.cc:407] Connection header received
0807 18:19:48.698212 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:48.698218 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:48.698300 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:48.698415 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:19:48.699248 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:48.700211 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:48.702144 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:48.703496 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:48.703709 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:48.703786 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:48.704403 (+ 617us) server_negotiation.cc:299] Negotiation successful
0807 18:19:48.704547 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260807 18:19:48.704820 14926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:48.696415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.110.2:45969 (local address 127.17.110.1:40505)
0807 18:19:48.696596 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:48.696603 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:48.697354 (+ 751us) server_negotiation.cc:407] Connection header received
0807 18:19:48.697781 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:48.697789 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:48.697895 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:48.698009 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:19:48.698756 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:48.699673 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:48.702282 (+ 2609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:48.703627 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:48.703782 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:48.703856 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:48.704403 (+ 547us) server_negotiation.cc:299] Negotiation successful
0807 18:19:48.704547 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 18:19:48.705052 14947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:48.696088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:40505 (local address 127.17.110.2:45969)
0807 18:19:48.697232 (+ 1144us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:48.697253 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:48.697564 (+ 311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:48.698082 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:48.698087 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:48.698160 (+ 73us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:19:48.698519 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:48.698531 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:48.699814 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:48.699821 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:48.701977 (+ 2156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:48.702000 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:48.703799 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:48.703814 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:48.704072 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:48.704230 (+ 158us) client_negotiation.cc:769] Sending connection context
0807 18:19:48.704332 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 18:19:48.704611 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":944,"mutex_wait_us":218,"thread_start_us":1142,"threads_started":1}
I20260807 18:19:48.705060 14945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:48.696183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.3:42259 (local address 127.17.110.2:58051)
0807 18:19:48.697685 (+ 1502us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:48.697705 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:19:48.697904 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:19:48.698513 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:19:48.698517 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:19:48.698542 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:19:48.699042 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:48.699054 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:48.700363 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:48.700368 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:48.701929 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:19:48.701943 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:19:48.703752 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:19:48.703761 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:19:48.704069 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:48.704230 (+ 161us) client_negotiation.cc:769] Sending connection context
0807 18:19:48.704332 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 18:19:48.704610 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1308,"mutex_wait_us":236,"thread_start_us":640,"threads_started":1}
I20260807 18:19:48.706413 14843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88bcba26644e4ec997ba62b58db3d68c" candidate_uuid: "f52ac5e88a72496282c89a3654ae556f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4eb90f2e49b74a9aaa6d5575fb8a8b08" is_pre_election: true
I20260807 18:19:48.706418 14470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88bcba26644e4ec997ba62b58db3d68c" candidate_uuid: "f52ac5e88a72496282c89a3654ae556f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5551a0d5d1ec44b9873902f083d1a00d" is_pre_election: true
I20260807 18:19:48.707098 14470 raft_consensus.cc:2454] T 88bcba26644e4ec997ba62b58db3d68c P 5551a0d5d1ec44b9873902f083d1a00d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f52ac5e88a72496282c89a3654ae556f in term 0.
I20260807 18:19:48.707098 14843 raft_consensus.cc:2454] T 88bcba26644e4ec997ba62b58db3d68c P 4eb90f2e49b74a9aaa6d5575fb8a8b08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f52ac5e88a72496282c89a3654ae556f in term 0.
I20260807 18:19:48.707897 14559 leader_election.cc:302] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4eb90f2e49b74a9aaa6d5575fb8a8b08, 5551a0d5d1ec44b9873902f083d1a00d, f52ac5e88a72496282c89a3654ae556f; no voters:
I20260807 18:19:48.708380 14944 raft_consensus.cc:2781] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:19:48.708546 14944 raft_consensus.cc:482] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:19:48.708653 14944 raft_consensus.cc:3037] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:19:48.710980 14944 raft_consensus.cc:504] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f52ac5e88a72496282c89a3654ae556f" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 41829 } } peers { permanent_uuid: "4eb90f2e49b74a9aaa6d5575fb8a8b08" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 42259 } } peers { permanent_uuid: "5551a0d5d1ec44b9873902f083d1a00d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 40505 } }
I20260807 18:19:48.711618 14944 leader_election.cc:288] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f [CANDIDATE]: Term 1 election: Requested vote from peers 4eb90f2e49b74a9aaa6d5575fb8a8b08 (127.17.110.3:42259), 5551a0d5d1ec44b9873902f083d1a00d (127.17.110.1:40505)
I20260807 18:19:48.712246 14843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88bcba26644e4ec997ba62b58db3d68c" candidate_uuid: "f52ac5e88a72496282c89a3654ae556f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4eb90f2e49b74a9aaa6d5575fb8a8b08"
I20260807 18:19:48.712473 14843 raft_consensus.cc:3037] T 88bcba26644e4ec997ba62b58db3d68c P 4eb90f2e49b74a9aaa6d5575fb8a8b08 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:19:48.712657 14470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88bcba26644e4ec997ba62b58db3d68c" candidate_uuid: "f52ac5e88a72496282c89a3654ae556f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5551a0d5d1ec44b9873902f083d1a00d"
I20260807 18:19:48.712889 14470 raft_consensus.cc:3037] T 88bcba26644e4ec997ba62b58db3d68c P 5551a0d5d1ec44b9873902f083d1a00d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:19:48.715946 14843 raft_consensus.cc:2454] T 88bcba26644e4ec997ba62b58db3d68c P 4eb90f2e49b74a9aaa6d5575fb8a8b08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f52ac5e88a72496282c89a3654ae556f in term 1.
I20260807 18:19:48.716169 14470 raft_consensus.cc:2454] T 88bcba26644e4ec997ba62b58db3d68c P 5551a0d5d1ec44b9873902f083d1a00d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f52ac5e88a72496282c89a3654ae556f in term 1.
I20260807 18:19:48.716532 14559 leader_election.cc:302] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4eb90f2e49b74a9aaa6d5575fb8a8b08, f52ac5e88a72496282c89a3654ae556f; no voters:
I20260807 18:19:48.716949 14944 raft_consensus.cc:2781] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:19:48.717428 14944 raft_consensus.cc:686] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f [term 1 LEADER]: Becoming Leader. State: Replica: f52ac5e88a72496282c89a3654ae556f, State: Running, Role: LEADER
W20260807 18:19:48.717679 14902 tablet.cc:2369] T 88bcba26644e4ec997ba62b58db3d68c P 4eb90f2e49b74a9aaa6d5575fb8a8b08: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:19:48.718039 14944 consensus_queue.cc:237] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f52ac5e88a72496282c89a3654ae556f" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 41829 } } peers { permanent_uuid: "4eb90f2e49b74a9aaa6d5575fb8a8b08" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 42259 } } peers { permanent_uuid: "5551a0d5d1ec44b9873902f083d1a00d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 40505 } }
I20260807 18:19:48.723384 14246 catalog_manager.cc:5495] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f reported cstate change: term changed from 0 to 1, leader changed from <none> to f52ac5e88a72496282c89a3654ae556f (127.17.110.2). New cstate: current_term: 1 leader_uuid: "f52ac5e88a72496282c89a3654ae556f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f52ac5e88a72496282c89a3654ae556f" member_type: VOTER last_known_addr { host: "127.17.110.2" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4eb90f2e49b74a9aaa6d5575fb8a8b08" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 42259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5551a0d5d1ec44b9873902f083d1a00d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260807 18:19:48.749979 14923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:48.744000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48622 (local address 127.17.110.2:41829)
0807 18:19:48.744239 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:48.744244 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:48.744268 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:19:48.744371 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:48.744375 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:48.744473 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:48.744664 (+ 191us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:19:48.745148 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:48.745952 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:48.747646 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:48.748182 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:48.748294 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:48.748869 (+ 575us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:19:48.749424 (+ 555us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:19:48.749556 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:48.749680 (+ 124us) server_negotiation.cc:299] Negotiation successful
0807 18:19:48.749752 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 18:19:48.758685 14843 raft_consensus.cc:1261] T 88bcba26644e4ec997ba62b58db3d68c P 4eb90f2e49b74a9aaa6d5575fb8a8b08 [term 1 FOLLOWER]: Refusing update from remote peer f52ac5e88a72496282c89a3654ae556f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:19:48.758759 14470 raft_consensus.cc:1261] T 88bcba26644e4ec997ba62b58db3d68c P 5551a0d5d1ec44b9873902f083d1a00d [term 1 FOLLOWER]: Refusing update from remote peer f52ac5e88a72496282c89a3654ae556f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:19:48.759896 14948 consensus_queue.cc:1035] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5551a0d5d1ec44b9873902f083d1a00d" member_type: VOTER last_known_addr { host: "127.17.110.1" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:19:48.760351 14944 consensus_queue.cc:1035] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4eb90f2e49b74a9aaa6d5575fb8a8b08" member_type: VOTER last_known_addr { host: "127.17.110.3" port: 42259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:19:48.771575 14952 mvcc.cc:205] Tried to move back new op lower bound from 7315975326744399872 to 7315975326593847296. Current Snapshot: MvccSnapshot[applied={T|T < 7315975326744399872}]
I20260807 18:19:48.773262 14953 mvcc.cc:205] Tried to move back new op lower bound from 7315975326744399872 to 7315975326593847296. Current Snapshot: MvccSnapshot[applied={T|T < 7315975326744399872}]
I20260807 18:19:48.773941 14954 mvcc.cc:205] Tried to move back new op lower bound from 7315975326744399872 to 7315975326593847296. Current Snapshot: MvccSnapshot[applied={T|T < 7315975326744399872}]
I20260807 18:19:48.788604 14246 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:19:48.788895 14246 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:19:48.793366 14246 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a2a7d4ded6b44afe97b3aee792e61994: Sending DeleteTablet for 3 replicas of tablet 88bcba26644e4ec997ba62b58db3d68c
I20260807 18:19:48.794770 14618 tablet_service.cc:1514] Processing DeleteTablet for tablet 88bcba26644e4ec997ba62b58db3d68c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:19:48 UTC) from {username='slave'} at 127.0.0.1:48614
I20260807 18:19:48.794983 14435 tablet_service.cc:1514] Processing DeleteTablet for tablet 88bcba26644e4ec997ba62b58db3d68c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:19:48 UTC) from {username='slave'} at 127.0.0.1:38382
I20260807 18:19:48.795629 14806 tablet_service.cc:1514] Processing DeleteTablet for tablet 88bcba26644e4ec997ba62b58db3d68c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:19:48 UTC) from {username='slave'} at 127.0.0.1:55978
I20260807 18:19:48.795877 14963 tablet_replica.cc:330] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f: stopping tablet replica
I20260807 18:19:48.796516 14963 raft_consensus.cc:2229] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:19:48.797565 14963 raft_consensus.cc:2258] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:19:48.800328 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 14335
I20260807 18:19:48.803791 14963 ts_tablet_manager.cc:1905] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:19:48.811125 14965 tablet_replica.cc:330] T 88bcba26644e4ec997ba62b58db3d68c P 4eb90f2e49b74a9aaa6d5575fb8a8b08: stopping tablet replica
I20260807 18:19:48.811863 14965 raft_consensus.cc:2229] T 88bcba26644e4ec997ba62b58db3d68c P 4eb90f2e49b74a9aaa6d5575fb8a8b08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:19:48.812604 14965 raft_consensus.cc:2258] T 88bcba26644e4ec997ba62b58db3d68c P 4eb90f2e49b74a9aaa6d5575fb8a8b08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:19:48.815287 14965 ts_tablet_manager.cc:1905] T 88bcba26644e4ec997ba62b58db3d68c P 4eb90f2e49b74a9aaa6d5575fb8a8b08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:19:48.820572 14964 tablet_replica.cc:330] T 88bcba26644e4ec997ba62b58db3d68c P 5551a0d5d1ec44b9873902f083d1a00d: stopping tablet replica
I20260807 18:19:48.821269 14964 raft_consensus.cc:2229] T 88bcba26644e4ec997ba62b58db3d68c P 5551a0d5d1ec44b9873902f083d1a00d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:19:48.821979 14964 raft_consensus.cc:2258] T 88bcba26644e4ec997ba62b58db3d68c P 5551a0d5d1ec44b9873902f083d1a00d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:19:48.823614 14965 ts_tablet_manager.cc:1918] T 88bcba26644e4ec997ba62b58db3d68c P 4eb90f2e49b74a9aaa6d5575fb8a8b08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:19:48.823864 14965 log.cc:1196] T 88bcba26644e4ec997ba62b58db3d68c P 4eb90f2e49b74a9aaa6d5575fb8a8b08: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-2/wal/wals/88bcba26644e4ec997ba62b58db3d68c
I20260807 18:19:48.824534 14965 ts_tablet_manager.cc:1939] T 88bcba26644e4ec997ba62b58db3d68c P 4eb90f2e49b74a9aaa6d5575fb8a8b08: Deleting consensus metadata
I20260807 18:19:48.825520 14964 ts_tablet_manager.cc:1905] T 88bcba26644e4ec997ba62b58db3d68c P 5551a0d5d1ec44b9873902f083d1a00d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:19:48.826879 14230 catalog_manager.cc:4841] TS 4eb90f2e49b74a9aaa6d5575fb8a8b08 (127.17.110.3:42259): tablet 88bcba26644e4ec997ba62b58db3d68c (table test-table [id=0e2694e20c6045c4807d476f3baafc92]) successfully deleted
I20260807 18:19:48.832193 14926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:48.801722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38388 (local address 127.17.110.1:40505)
0807 18:19:48.801915 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:48.801922 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:48.823027 (+ 21105us) server_negotiation.cc:407] Connection header received
0807 18:19:48.823259 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:48.823264 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:48.823384 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:48.823493 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:48.824230 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:48.825340 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:48.827711 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:48.828382 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:48.828484 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:48.830146 (+ 1662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:48.830160 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:48.830181 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:19:48.831130 (+ 949us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:48.831196 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:19:48.831230 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:48.831368 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:48.831627 (+ 259us) server_negotiation.cc:299] Negotiation successful
0807 18:19:48.831753 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:19:48.833374 14964 ts_tablet_manager.cc:1918] T 88bcba26644e4ec997ba62b58db3d68c P 5551a0d5d1ec44b9873902f083d1a00d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:19:48.833460 14520 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:19:48.834410 14963 ts_tablet_manager.cc:1918] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:19:48.834679 14963 log.cc:1196] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-1/wal/wals/88bcba26644e4ec997ba62b58db3d68c
I20260807 18:19:48.835461 14963 ts_tablet_manager.cc:1939] T 88bcba26644e4ec997ba62b58db3d68c P f52ac5e88a72496282c89a3654ae556f: Deleting consensus metadata
I20260807 18:19:48.836385 14964 log.cc:1196] T 88bcba26644e4ec997ba62b58db3d68c P 5551a0d5d1ec44b9873902f083d1a00d: Deleting WAL directory at /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126429568977-17848-0/minicluster-data/ts-0/wal/wals/88bcba26644e4ec997ba62b58db3d68c
I20260807 18:19:48.837730 14230 catalog_manager.cc:4841] TS f52ac5e88a72496282c89a3654ae556f (127.17.110.2:41829): tablet 88bcba26644e4ec997ba62b58db3d68c (table test-table [id=0e2694e20c6045c4807d476f3baafc92]) successfully deleted
I20260807 18:19:48.948773 14964 ts_tablet_manager.cc:1939] T 88bcba26644e4ec997ba62b58db3d68c P 5551a0d5d1ec44b9873902f083d1a00d: Deleting consensus metadata
I20260807 18:19:48.949172 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 14335
I20260807 18:19:48.970480 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 14532
W20260807 18:19:48.970395 14229 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.110.1:40505 (error 108)
I20260807 18:19:48.982681 14922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:48.982180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:40505 (local address 127.0.0.1:38404)
0807 18:19:48.982346 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:48.982487 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 18:19:48.983302 14229 catalog_manager.cc:4584] TS 5551a0d5d1ec44b9873902f083d1a00d (127.17.110.1:40505): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 88bcba26644e4ec997ba62b58db3d68c: Network error: Client connection negotiation failed: client connection to 127.17.110.1:40505: connect: Connection refused (error 111)
I20260807 18:19:48.983759 14923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:48.972308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48626 (local address 127.17.110.2:41829)
0807 18:19:48.972549 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:48.972557 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:48.972583 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:19:48.972667 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:48.972674 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:48.972808 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:48.972935 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:48.973841 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:48.975680 (+ 1839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:48.978308 (+ 2628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:48.979310 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:48.979488 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:48.981303 (+ 1815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:48.981315 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:48.981333 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:19:48.982287 (+ 954us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:48.982351 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:19:48.982387 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:48.982497 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:48.983249 (+ 752us) server_negotiation.cc:299] Negotiation successful
0807 18:19:48.983369 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 18:19:48.984649 14705 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:19:49.037429 14922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:49.036806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:40505 (local address 127.0.0.1:38412)
0807 18:19:49.037091 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:49.037213 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260807 18:19:49.038812 14922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:49.038289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:40505 (local address 127.0.0.1:38426)
0807 18:19:49.038470 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:49.038594 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 18:19:49.099622 14922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:49.099024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:40505 (local address 127.0.0.1:38428)
0807 18:19:49.099252 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:49.099421 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 18:19:49.101303 14922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:49.100754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:40505 (local address 127.0.0.1:38438)
0807 18:19:49.100946 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:49.101122 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 18:19:49.101928 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 14532
I20260807 18:19:49.131175 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 14715
I20260807 18:19:49.138417 14927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:49.132151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55984 (local address 127.17.110.3:42259)
0807 18:19:49.132364 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:49.132370 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:49.132394 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:19:49.132501 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:49.132505 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:49.132601 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:49.132719 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:49.133119 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:49.133854 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:49.135212 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:49.135613 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:49.135765 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:49.136952 (+ 1187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:49.136962 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:49.136979 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:19:49.137600 (+ 621us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:49.137663 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:19:49.137688 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:49.137820 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:49.137953 (+ 133us) server_negotiation.cc:299] Negotiation successful
0807 18:19:49.138080 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 18:19:49.139240 14896 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:19:49.182092 14922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:49.181403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:40505 (local address 127.0.0.1:38450)
0807 18:19:49.181736 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:49.181860 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260807 18:19:49.183471 14922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:49.182981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.1:40505 (local address 127.0.0.1:38462)
0807 18:19:49.183223 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0807 18:19:49.183319 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.1:40505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260807 18:19:49.231978 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 14715
I20260807 18:19:49.256295 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 14199
I20260807 18:19:49.257556 14286 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:19:49.370189 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 14199
2026-08-07T18:19:49Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3124 ms)
[----------] 6 tests from EncryptionPolicyTest (25211 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:19:49 dist-test-slave-39l8 krb5kdc[14988](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:19:49 dist-test-slave-39l8 krb5kdc[14988](info): set up 2 sockets
Aug 07 18:19:49 dist-test-slave-39l8 krb5kdc[14988](info): commencing operation
krb5kdc: starting...
W20260807 18:19:50.790946 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.359s user 0.000s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:19:51 dist-test-slave-39l8 krb5kdc[14988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:19:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:19:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:19:51.156457 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38709
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:19:51.524991 15004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:19:51.525336 15004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:19:51.525481 15004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:19:51.533102 15004 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:19:51.533236 15004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:19:51.533334 15004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:19:51.533438 15004 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:19:51.533514 15004 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:19:51.543814 15004 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38709
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:19:51.545265 15004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:19:51.546746 15004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:19:51.559180 15013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:51.559223 15018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:51.559088 15014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:19:51.560242 15004 server_base.cc:1034] running on GCE node
I20260807 18:19:51.561121 15004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:19:51.562686 15004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:19:51.563995 15004 hybrid_clock.cc:648] HybridClock initialized: now 1786126791563941 us; error 43 us; skew 500 ppm
I20260807 18:19:51.564391 15004 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:19:51 dist-test-slave-39l8 krb5kdc[14988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126791, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:19:51.567734 15004 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:19:51.569414 15004 webserver.cc:460] Webserver started at http://127.0.0.1:45721/ using document root <none> and password file <none>
I20260807 18:19:51.570142 15004 fs_manager.cc:359] Metadata directory not provided
I20260807 18:19:51.570258 15004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:19:51.570528 15004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:19:51.573784 15004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "9fba074350594cbf8d3f2700f0215d25"
format_stamp: "Formatted at 2026-08-07 18:19:51 on dist-test-slave-39l8"
server_key: "60a39c1436eb4210c7a5b20e7ee707e8"
server_key_iv: "86370a8084a8eff655627e8b78cf20fd"
server_key_version: "encryptionkey@0"
I20260807 18:19:51.574572 15004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "9fba074350594cbf8d3f2700f0215d25"
format_stamp: "Formatted at 2026-08-07 18:19:51 on dist-test-slave-39l8"
server_key: "60a39c1436eb4210c7a5b20e7ee707e8"
server_key_iv: "86370a8084a8eff655627e8b78cf20fd"
server_key_version: "encryptionkey@0"
I20260807 18:19:51.580292 15004 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 18:19:51.584322 15028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:51.585980 15004 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:19:51.586259 15004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "9fba074350594cbf8d3f2700f0215d25"
format_stamp: "Formatted at 2026-08-07 18:19:51 on dist-test-slave-39l8"
server_key: "60a39c1436eb4210c7a5b20e7ee707e8"
server_key_iv: "86370a8084a8eff655627e8b78cf20fd"
server_key_version: "encryptionkey@0"
I20260807 18:19:51.586491 15004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:19:51.617569 15004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:19:51.622529 15004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:19:51.622931 15004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:19:51.644055 15004 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38709
I20260807 18:19:51.644095 15105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38709 every 8 connection(s)
I20260807 18:19:51.645561 15004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:19:51.649787 15106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:19:51.652046 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 15004
I20260807 18:19:51.652340 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:19:51.653024 17848 external_mini_cluster.cc:1428] Setting key 4a89b63e1cc1683aed8f982454cd2dc2
I20260807 18:19:51.663471 15106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fba074350594cbf8d3f2700f0215d25: Bootstrap starting.
I20260807 18:19:51.667168 15106 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9fba074350594cbf8d3f2700f0215d25: Neither blocks nor log segments found. Creating new log.
I20260807 18:19:51.668391 15106 log.cc:824] T 00000000000000000000000000000000 P 9fba074350594cbf8d3f2700f0215d25: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:19:51 dist-test-slave-39l8 krb5kdc[14988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 18:19:51.672739 15106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fba074350594cbf8d3f2700f0215d25: No bootstrap required, opened a new log
I20260807 18:19:51.679447 15111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:51.655428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55769 (local address 127.0.0.1:38709)
0807 18:19:51.656423 (+ 995us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:51.656436 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:51.656483 (+ 47us) server_negotiation.cc:407] Connection header received
0807 18:19:51.658354 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:51.658382 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:51.658914 (+ 532us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:51.659346 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:51.660920 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:51.662328 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:51.664235 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:51.664969 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:51.665209 (+ 240us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:51.670180 (+ 4971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:51.670199 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:51.670218 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:19:51.670266 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:51.673958 (+ 3692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:51.675303 (+ 1345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:51.675312 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:51.675321 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:51.675543 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:51.676201 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:51.676205 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:51.676207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:51.676839 (+ 632us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:51.677137 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:51.677781 (+ 644us) server_negotiation.cc:299] Negotiation successful
0807 18:19:51.678167 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":715,"thread_start_us":392,"threads_started":1}
I20260807 18:19:51.683753 15106 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fba074350594cbf8d3f2700f0215d25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fba074350594cbf8d3f2700f0215d25" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38709 } }
I20260807 18:19:51.684333 15106 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9fba074350594cbf8d3f2700f0215d25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:19:51.684459 15106 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fba074350594cbf8d3f2700f0215d25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fba074350594cbf8d3f2700f0215d25, State: Initialized, Role: FOLLOWER
I20260807 18:19:51.684648 17848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:19:51.685249 15106 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fba074350594cbf8d3f2700f0215d25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fba074350594cbf8d3f2700f0215d25" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38709 } }
I20260807 18:19:51.685554 15106 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fba074350594cbf8d3f2700f0215d25 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:19:51.685707 15106 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fba074350594cbf8d3f2700f0215d25 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:19:51.685900 15106 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fba074350594cbf8d3f2700f0215d25 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:19:51.689136 15106 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fba074350594cbf8d3f2700f0215d25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fba074350594cbf8d3f2700f0215d25" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38709 } }
I20260807 18:19:51.689818 15106 leader_election.cc:302] T 00000000000000000000000000000000 P 9fba074350594cbf8d3f2700f0215d25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fba074350594cbf8d3f2700f0215d25; no voters:
I20260807 18:19:51.690732 15106 leader_election.cc:288] T 00000000000000000000000000000000 P 9fba074350594cbf8d3f2700f0215d25 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:19:51.690861 15121 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fba074350594cbf8d3f2700f0215d25 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:19:51.697710 15106 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fba074350594cbf8d3f2700f0215d25 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:19:51.698098 15121 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fba074350594cbf8d3f2700f0215d25 [term 1 LEADER]: Becoming Leader. State: Replica: 9fba074350594cbf8d3f2700f0215d25, State: Running, Role: LEADER
I20260807 18:19:51.698913 15121 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fba074350594cbf8d3f2700f0215d25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fba074350594cbf8d3f2700f0215d25" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38709 } }
I20260807 18:19:51.703316 15125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fba074350594cbf8d3f2700f0215d25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9fba074350594cbf8d3f2700f0215d25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fba074350594cbf8d3f2700f0215d25" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38709 } } }
I20260807 18:19:51.703914 15125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fba074350594cbf8d3f2700f0215d25 [sys.catalog]: This master's current role is: LEADER
I20260807 18:19:51.708048 15126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fba074350594cbf8d3f2700f0215d25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fba074350594cbf8d3f2700f0215d25. Latest consensus state: current_term: 1 leader_uuid: "9fba074350594cbf8d3f2700f0215d25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fba074350594cbf8d3f2700f0215d25" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38709 } } }
I20260807 18:19:51.708510 15126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fba074350594cbf8d3f2700f0215d25 [sys.catalog]: This master's current role is: LEADER
I20260807 18:19:51.711054 15111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:51.690520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60117 (local address 127.0.0.1:38709)
0807 18:19:51.691209 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:51.691219 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:51.691907 (+ 688us) server_negotiation.cc:407] Connection header received
0807 18:19:51.692160 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:51.692164 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:51.692232 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:51.692354 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:51.693903 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:51.694816 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:51.696512 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:51.696986 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:51.697083 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:51.700365 (+ 3282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:51.700370 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:51.700374 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:19:51.700412 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:51.704719 (+ 4307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:51.706300 (+ 1581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:51.706309 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:51.706316 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:51.706538 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:51.707439 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:51.707447 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:51.707453 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:51.708030 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:51.709868 (+ 1838us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:51.710147 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 18:19:51.710443 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413}
I20260807 18:19:51.716611 15131 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:19:51.724735 15131 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:19:51.736491 15131 catalog_manager.cc:1347] Generated new cluster ID: e94d7bf38a6845d3b2661d7dc3472954
I20260807 18:19:51.736644 15131 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:19:51.757651 15131 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:19:51.759819 15131 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:19:51.774323 15131 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9fba074350594cbf8d3f2700f0215d25: Generated new TSK 0
I20260807 18:19:51.775616 15131 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:19:51.792313 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 15004
I20260807 18:19:51.793943 15101 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:19:51.889590 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 15004
W20260807 18:19:51.917241 14983 connection.cc:535] client connection to 127.0.0.1:38709 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38709): Connection reset by peer (error 104)
2026-08-07T18:19:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2507 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:19:52 dist-test-slave-39l8 krb5kdc[15154](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:19:52 dist-test-slave-39l8 krb5kdc[15154](info): set up 2 sockets
Aug 07 18:19:52 dist-test-slave-39l8 krb5kdc[15154](info): commencing operation
krb5kdc: starting...
W20260807 18:19:53.242779 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.305s user 0.003s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:19:53 dist-test-slave-39l8 krb5kdc[15154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:19:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:19:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:19:53.592542 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35797
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:19:53.954689 15171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:19:53.955127 15171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:19:53.955229 15171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:19:53.963045 15171 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:19:53.963191 15171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:19:53.963240 15171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:19:53.963312 15171 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:19:53.963359 15171 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:19:53.973831 15171 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35797
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:19:53.975405 15171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:19:53.976740 15171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:19:53.989475 15182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:53.989755 15186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:53.989518 15183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:19:53.990768 15171 server_base.cc:1034] running on GCE node
I20260807 18:19:53.991674 15171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:19:53.993346 15171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:19:53.994616 15171 hybrid_clock.cc:648] HybridClock initialized: now 1786126793994558 us; error 61 us; skew 500 ppm
I20260807 18:19:53.995083 15171 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:19:54 dist-test-slave-39l8 krb5kdc[15154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126794, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:19:54.004307 15171 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:19:54.006292 15171 webserver.cc:460] Webserver started at http://127.0.0.1:39669/ using document root <none> and password file <none>
I20260807 18:19:54.007205 15171 fs_manager.cc:359] Metadata directory not provided
I20260807 18:19:54.007359 15171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:19:54.007743 15171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:19:54.011212 15171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "503bf6fef1594337af6547f17f54f036"
format_stamp: "Formatted at 2026-08-07 18:19:54 on dist-test-slave-39l8"
server_key: "295fdbad1d15c7ed1e3230f942a0720e"
server_key_iv: "68ce747916f59b376591b67c1242196b"
server_key_version: "encryptionkey@0"
I20260807 18:19:54.012179 15171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "503bf6fef1594337af6547f17f54f036"
format_stamp: "Formatted at 2026-08-07 18:19:54 on dist-test-slave-39l8"
server_key: "295fdbad1d15c7ed1e3230f942a0720e"
server_key_iv: "68ce747916f59b376591b67c1242196b"
server_key_version: "encryptionkey@0"
I20260807 18:19:54.019093 15171 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 18:19:54.024677 15195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:54.026479 15171 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:19:54.026825 15171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "503bf6fef1594337af6547f17f54f036"
format_stamp: "Formatted at 2026-08-07 18:19:54 on dist-test-slave-39l8"
server_key: "295fdbad1d15c7ed1e3230f942a0720e"
server_key_iv: "68ce747916f59b376591b67c1242196b"
server_key_version: "encryptionkey@0"
I20260807 18:19:54.027159 15171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:19:54.041738 15171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:19:54.046844 15171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:19:54.047178 15171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:19:54.068085 15171 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35797
I20260807 18:19:54.068163 15276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35797 every 8 connection(s)
I20260807 18:19:54.069692 15171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:19:54.074110 15277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:19:54.077687 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 15171
I20260807 18:19:54.077919 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:19:54.078480 17848 external_mini_cluster.cc:1428] Setting key 0375f187373fedc734181ad3688a5824
I20260807 18:19:54.088439 15277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 503bf6fef1594337af6547f17f54f036: Bootstrap starting.
I20260807 18:19:54.093252 15277 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 503bf6fef1594337af6547f17f54f036: Neither blocks nor log segments found. Creating new log.
I20260807 18:19:54.094892 15277 log.cc:824] T 00000000000000000000000000000000 P 503bf6fef1594337af6547f17f54f036: Log is configured to *not* fsync() on all Append() calls
I20260807 18:19:54.094973 15282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:54.081040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49239 (local address 127.0.0.1:35797)
0807 18:19:54.082020 (+ 980us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:54.082035 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:54.082087 (+ 52us) server_negotiation.cc:407] Connection header received
0807 18:19:54.084092 (+ 2005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:54.084119 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:54.084600 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:54.085073 (+ 473us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:54.085719 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:54.087209 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:54.088679 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:54.089871 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:54.090113 (+ 242us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:54.091485 (+ 1372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:54.091498 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:54.091518 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:19:54.092387 (+ 869us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:54.092498 (+ 111us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:19:54.092536 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:54.092762 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:54.093106 (+ 344us) server_negotiation.cc:299] Negotiation successful
0807 18:19:54.093334 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":710,"thread_start_us":438,"threads_started":1}
I20260807 18:19:54.098513 15277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 503bf6fef1594337af6547f17f54f036: No bootstrap required, opened a new log
I20260807 18:19:54.099795 17848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:19:54.109686 15277 raft_consensus.cc:348] T 00000000000000000000000000000000 P 503bf6fef1594337af6547f17f54f036 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "503bf6fef1594337af6547f17f54f036" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35797 } }
I20260807 18:19:54.110349 15277 raft_consensus.cc:374] T 00000000000000000000000000000000 P 503bf6fef1594337af6547f17f54f036 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:19:54.110513 15277 raft_consensus.cc:729] T 00000000000000000000000000000000 P 503bf6fef1594337af6547f17f54f036 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 503bf6fef1594337af6547f17f54f036, State: Initialized, Role: FOLLOWER
I20260807 18:19:54.111330 15277 consensus_queue.cc:260] T 00000000000000000000000000000000 P 503bf6fef1594337af6547f17f54f036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "503bf6fef1594337af6547f17f54f036" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35797 } }
I20260807 18:19:54.111662 15277 raft_consensus.cc:388] T 00000000000000000000000000000000 P 503bf6fef1594337af6547f17f54f036 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:19:54.111850 15277 raft_consensus.cc:482] T 00000000000000000000000000000000 P 503bf6fef1594337af6547f17f54f036 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:19:54.112103 15277 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 503bf6fef1594337af6547f17f54f036 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:19:54.112891 15282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:54.104736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34779 (local address 127.0.0.1:35797)
0807 18:19:54.104981 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:54.104989 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:54.105413 (+ 424us) server_negotiation.cc:407] Connection header received
0807 18:19:54.105598 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:54.105604 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:54.105705 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:54.105866 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:54.106374 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:54.107804 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:54.109361 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:54.110076 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:54.110256 (+ 180us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:54.111846 (+ 1590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:54.111854 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:54.111861 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:19:54.111893 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:54.111951 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:19:54.111976 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:54.112173 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:54.112415 (+ 242us) server_negotiation.cc:299] Negotiation successful
0807 18:19:54.112494 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 18:19:54.115312 15277 raft_consensus.cc:504] T 00000000000000000000000000000000 P 503bf6fef1594337af6547f17f54f036 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "503bf6fef1594337af6547f17f54f036" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35797 } }
I20260807 18:19:54.115895 15277 leader_election.cc:302] T 00000000000000000000000000000000 P 503bf6fef1594337af6547f17f54f036 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 503bf6fef1594337af6547f17f54f036; no voters:
I20260807 18:19:54.116514 15277 leader_election.cc:288] T 00000000000000000000000000000000 P 503bf6fef1594337af6547f17f54f036 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:19:54.116719 15292 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 503bf6fef1594337af6547f17f54f036 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:19:54.117540 15292 raft_consensus.cc:686] T 00000000000000000000000000000000 P 503bf6fef1594337af6547f17f54f036 [term 1 LEADER]: Becoming Leader. State: Replica: 503bf6fef1594337af6547f17f54f036, State: Running, Role: LEADER
I20260807 18:19:54.118251 15292 consensus_queue.cc:237] T 00000000000000000000000000000000 P 503bf6fef1594337af6547f17f54f036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "503bf6fef1594337af6547f17f54f036" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35797 } }
I20260807 18:19:54.119035 15277 sys_catalog.cc:564] T 00000000000000000000000000000000 P 503bf6fef1594337af6547f17f54f036 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:19:54.121341 15295 sys_catalog.cc:455] T 00000000000000000000000000000000 P 503bf6fef1594337af6547f17f54f036 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 503bf6fef1594337af6547f17f54f036. Latest consensus state: current_term: 1 leader_uuid: "503bf6fef1594337af6547f17f54f036" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "503bf6fef1594337af6547f17f54f036" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35797 } } }
I20260807 18:19:54.121548 15293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 503bf6fef1594337af6547f17f54f036 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "503bf6fef1594337af6547f17f54f036" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "503bf6fef1594337af6547f17f54f036" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35797 } } }
I20260807 18:19:54.121855 15293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 503bf6fef1594337af6547f17f54f036 [sys.catalog]: This master's current role is: LEADER
I20260807 18:19:54.121855 15295 sys_catalog.cc:458] T 00000000000000000000000000000000 P 503bf6fef1594337af6547f17f54f036 [sys.catalog]: This master's current role is: LEADER
I20260807 18:19:54.122946 15302 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:19:54.134501 15302 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:19:54.143970 15302 catalog_manager.cc:1347] Generated new cluster ID: a373a4f245724ff1869b2406251cf844
I20260807 18:19:54.144126 15302 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:19:54.166841 15302 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:19:54.168880 15302 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:19:54.177289 15302 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 503bf6fef1594337af6547f17f54f036: Generated new TSK 0
I20260807 18:19:54.178354 15302 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:19:54.192655 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 15171
W20260807 18:19:54.194092 15270 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:49239
W20260807 18:19:54.195047 17848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 15171: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:19:54.195329 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 15171
W20260807 18:19:54.222563 15152 connection.cc:535] client connection to 127.0.0.1:35797 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35797): Connection reset by peer (error 104)
2026-08-07T18:19:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2304 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:19:54 dist-test-slave-39l8 krb5kdc[15326](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:19:54 dist-test-slave-39l8 krb5kdc[15326](info): set up 2 sockets
Aug 07 18:19:54 dist-test-slave-39l8 krb5kdc[15326](info): commencing operation
krb5kdc: starting...
W20260807 18:19:55.542153 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.300s user 0.003s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:19:55 dist-test-slave-39l8 krb5kdc[15326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:19:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:19:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:19:55.900462 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45107
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45107
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:19:56.282481 15352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:19:56.282939 15352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:19:56.283167 15352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:19:56.291544 15352 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:19:56.291692 15352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:19:56.291769 15352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:19:56.291883 15352 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:19:56.291975 15352 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:19:56.302597 15352 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45107
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:19:56.304083 15352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:19:56.305532 15352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:19:56.319056 15359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:56.319093 15360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:56.319785 15364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:19:56.320456 15352 server_base.cc:1034] running on GCE node
I20260807 18:19:56.321484 15352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:19:56.323283 15352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:19:56.324576 15352 hybrid_clock.cc:648] HybridClock initialized: now 1786126796324517 us; error 59 us; skew 500 ppm
I20260807 18:19:56.325003 15352 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:19:56 dist-test-slave-39l8 krb5kdc[15326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126796, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:19:56.328950 15352 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:19:56.330993 15352 webserver.cc:460] Webserver started at http://127.0.0.1:41297/ using document root <none> and password file <none>
I20260807 18:19:56.331830 15352 fs_manager.cc:359] Metadata directory not provided
I20260807 18:19:56.332015 15352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:19:56.332394 15352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:19:56.336050 15352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "7deb17d5f1d9485a8070a95f8006ad4d"
format_stamp: "Formatted at 2026-08-07 18:19:56 on dist-test-slave-39l8"
server_key: "7db789d1f668da82b07f163bb142b35f"
server_key_iv: "ab688fedae5fbc5f3e20ff3934172088"
server_key_version: "encryptionkey@0"
I20260807 18:19:56.337006 15352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "7deb17d5f1d9485a8070a95f8006ad4d"
format_stamp: "Formatted at 2026-08-07 18:19:56 on dist-test-slave-39l8"
server_key: "7db789d1f668da82b07f163bb142b35f"
server_key_iv: "ab688fedae5fbc5f3e20ff3934172088"
server_key_version: "encryptionkey@0"
I20260807 18:19:56.343686 15352 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260807 18:19:56.348245 15373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:56.350010 15352 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260807 18:19:56.350301 15352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "7deb17d5f1d9485a8070a95f8006ad4d"
format_stamp: "Formatted at 2026-08-07 18:19:56 on dist-test-slave-39l8"
server_key: "7db789d1f668da82b07f163bb142b35f"
server_key_iv: "ab688fedae5fbc5f3e20ff3934172088"
server_key_version: "encryptionkey@0"
I20260807 18:19:56.350562 15352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:19:56.357314 15352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:19:56.362597 15352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:19:56.363070 15352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:19:56.383790 15352 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45107
I20260807 18:19:56.383837 15449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45107 every 8 connection(s)
I20260807 18:19:56.385447 15352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:19:56.387233 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 15352
I20260807 18:19:56.387427 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:19:56.387902 17848 external_mini_cluster.cc:1428] Setting key 579da3fbdc42f0a89a553c119b689975
I20260807 18:19:56.390321 15450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:19:56.399004 15455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:56.390192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41299 (local address 127.0.0.1:45107)
0807 18:19:56.391372 (+ 1180us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:56.391391 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:56.391817 (+ 426us) server_negotiation.cc:407] Connection header received
0807 18:19:56.394228 (+ 2411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:56.394264 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:56.394856 (+ 592us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:56.395429 (+ 573us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:56.396574 (+ 1145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:56.396587 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:56.396614 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:19:56.396633 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:56.396750 (+ 117us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:19:56.396779 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:56.397042 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:56.397222 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 18:19:56.397448 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":855,"thread_start_us":512,"threads_started":1}
I20260807 18:19:56.403419 17848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:19:56.408057 15450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7deb17d5f1d9485a8070a95f8006ad4d: Bootstrap starting.
I20260807 18:19:56.412925 15450 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7deb17d5f1d9485a8070a95f8006ad4d: Neither blocks nor log segments found. Creating new log.
I20260807 18:19:56.412925 15455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:56.409797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58349 (local address 127.0.0.1:45107)
0807 18:19:56.410028 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:56.410037 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:56.410635 (+ 598us) server_negotiation.cc:407] Connection header received
0807 18:19:56.410913 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:56.410917 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:56.410985 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:56.411088 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:56.412197 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:56.412201 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:56.412205 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:19:56.412221 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:56.412266 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:19:56.412282 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:56.412377 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:56.412549 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 18:19:56.412599 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:19:56.414554 15450 log.cc:824] T 00000000000000000000000000000000 P 7deb17d5f1d9485a8070a95f8006ad4d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:19:56.417747 15450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7deb17d5f1d9485a8070a95f8006ad4d: No bootstrap required, opened a new log
I20260807 18:19:56.427340 15450 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7deb17d5f1d9485a8070a95f8006ad4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7deb17d5f1d9485a8070a95f8006ad4d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45107 } }
I20260807 18:19:56.427836 15450 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7deb17d5f1d9485a8070a95f8006ad4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:19:56.427927 15450 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7deb17d5f1d9485a8070a95f8006ad4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7deb17d5f1d9485a8070a95f8006ad4d, State: Initialized, Role: FOLLOWER
I20260807 18:19:56.428517 15450 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7deb17d5f1d9485a8070a95f8006ad4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7deb17d5f1d9485a8070a95f8006ad4d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45107 } }
I20260807 18:19:56.428747 15450 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7deb17d5f1d9485a8070a95f8006ad4d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:19:56.428862 15450 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7deb17d5f1d9485a8070a95f8006ad4d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:19:56.429013 15450 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7deb17d5f1d9485a8070a95f8006ad4d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:19:56.431864 15450 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7deb17d5f1d9485a8070a95f8006ad4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7deb17d5f1d9485a8070a95f8006ad4d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45107 } }
I20260807 18:19:56.432668 15450 leader_election.cc:302] T 00000000000000000000000000000000 P 7deb17d5f1d9485a8070a95f8006ad4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7deb17d5f1d9485a8070a95f8006ad4d; no voters:
I20260807 18:19:56.433475 15450 leader_election.cc:288] T 00000000000000000000000000000000 P 7deb17d5f1d9485a8070a95f8006ad4d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:19:56.433661 15472 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7deb17d5f1d9485a8070a95f8006ad4d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:19:56.434484 15472 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7deb17d5f1d9485a8070a95f8006ad4d [term 1 LEADER]: Becoming Leader. State: Replica: 7deb17d5f1d9485a8070a95f8006ad4d, State: Running, Role: LEADER
I20260807 18:19:56.435169 15472 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7deb17d5f1d9485a8070a95f8006ad4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7deb17d5f1d9485a8070a95f8006ad4d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45107 } }
I20260807 18:19:56.436017 15450 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7deb17d5f1d9485a8070a95f8006ad4d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:19:56.438234 15473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7deb17d5f1d9485a8070a95f8006ad4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7deb17d5f1d9485a8070a95f8006ad4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7deb17d5f1d9485a8070a95f8006ad4d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45107 } } }
I20260807 18:19:56.438853 15473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7deb17d5f1d9485a8070a95f8006ad4d [sys.catalog]: This master's current role is: LEADER
I20260807 18:19:56.441147 15474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7deb17d5f1d9485a8070a95f8006ad4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7deb17d5f1d9485a8070a95f8006ad4d. Latest consensus state: current_term: 1 leader_uuid: "7deb17d5f1d9485a8070a95f8006ad4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7deb17d5f1d9485a8070a95f8006ad4d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45107 } } }
I20260807 18:19:56.441636 15474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7deb17d5f1d9485a8070a95f8006ad4d [sys.catalog]: This master's current role is: LEADER
I20260807 18:19:56.444177 15483 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:19:56.449190 15483 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:19:56.459908 15483 catalog_manager.cc:1347] Generated new cluster ID: 96d90ce08de845cfa0ed6030ea6650f7
I20260807 18:19:56.460101 15483 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:19:56.491741 15483 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:19:56.492251 15483 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:19:56.500765 15483 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7deb17d5f1d9485a8070a95f8006ad4d: Generated new TSK 0
I20260807 18:19:56.501744 15483 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:19:56.514566 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 15352
W20260807 18:19:56.516022 15444 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:41299
W20260807 18:19:56.516847 17848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 15352: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:19:56.517031 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 15352
2026-08-07T18:19:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2325 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:19:56 dist-test-slave-39l8 krb5kdc[15499](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:19:56 dist-test-slave-39l8 krb5kdc[15499](info): set up 2 sockets
Aug 07 18:19:56 dist-test-slave-39l8 krb5kdc[15499](info): commencing operation
krb5kdc: starting...
W20260807 18:19:57.871536 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.303s user 0.006s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:19:58 dist-test-slave-39l8 krb5kdc[15499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:19:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:19:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:19:58.241815 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43239
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:19:58.615309 15519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:19:58.615697 15519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:19:58.615880 15519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:19:58.623433 15519 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:19:58.623637 15519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:19:58.623757 15519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:19:58.623865 15519 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:19:58.623948 15519 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:19:58.634313 15519 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43239
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:19:58.635802 15519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:19:58.637262 15519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:19:58.650825 15528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:58.650877 15526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:19:58.650916 15525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:19:58.651826 15519 server_base.cc:1034] running on GCE node
I20260807 18:19:58.652679 15519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:19:58.654260 15519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:19:58.655485 15519 hybrid_clock.cc:648] HybridClock initialized: now 1786126798655416 us; error 61 us; skew 500 ppm
I20260807 18:19:58.655855 15519 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:19:58 dist-test-slave-39l8 krb5kdc[15499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126798, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:19:58.659554 15519 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:19:58.661361 15519 webserver.cc:460] Webserver started at http://127.0.0.1:34435/ using document root <none> and password file <none>
I20260807 18:19:58.662138 15519 fs_manager.cc:359] Metadata directory not provided
I20260807 18:19:58.662292 15519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:19:58.662639 15519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:19:58.665961 15519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "902934c956df4d969031e1c3cdf270fc"
format_stamp: "Formatted at 2026-08-07 18:19:58 on dist-test-slave-39l8"
server_key: "14c9aaa63c97257032a59a624fc310ee"
server_key_iv: "f7dbeb46b46135a084a7410b52a658b9"
server_key_version: "encryptionkey@0"
I20260807 18:19:58.666903 15519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "902934c956df4d969031e1c3cdf270fc"
format_stamp: "Formatted at 2026-08-07 18:19:58 on dist-test-slave-39l8"
server_key: "14c9aaa63c97257032a59a624fc310ee"
server_key_iv: "f7dbeb46b46135a084a7410b52a658b9"
server_key_version: "encryptionkey@0"
I20260807 18:19:58.672763 15519 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:19:58.676869 15541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:19:58.678423 15519 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:19:58.678642 15519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "902934c956df4d969031e1c3cdf270fc"
format_stamp: "Formatted at 2026-08-07 18:19:58 on dist-test-slave-39l8"
server_key: "14c9aaa63c97257032a59a624fc310ee"
server_key_iv: "f7dbeb46b46135a084a7410b52a658b9"
server_key_version: "encryptionkey@0"
I20260807 18:19:58.678912 15519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:19:58.699674 15519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:19:58.704937 15519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:19:58.705312 15519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:19:58.726034 15519 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43239
I20260807 18:19:58.726069 15611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43239 every 8 connection(s)
I20260807 18:19:58.727599 15519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:19:58.727936 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 15519
I20260807 18:19:58.728134 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:19:58.728613 17848 external_mini_cluster.cc:1428] Setting key 3ee3808c16bd0f5a188fb04865e93ac4
I20260807 18:19:58.732461 15612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 18:19:58 dist-test-slave-39l8 krb5kdc[15499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 18:19:58.744428 15612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 902934c956df4d969031e1c3cdf270fc: Bootstrap starting.
I20260807 18:19:58.748306 15612 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 902934c956df4d969031e1c3cdf270fc: Neither blocks nor log segments found. Creating new log.
I20260807 18:19:58.749593 15612 log.cc:824] T 00000000000000000000000000000000 P 902934c956df4d969031e1c3cdf270fc: Log is configured to *not* fsync() on all Append() calls
I20260807 18:19:58.753528 15612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 902934c956df4d969031e1c3cdf270fc: No bootstrap required, opened a new log
I20260807 18:19:58.753563 15617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:58.730694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59311 (local address 127.0.0.1:43239)
0807 18:19:58.731607 (+ 913us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:58.731620 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:58.731663 (+ 43us) server_negotiation.cc:407] Connection header received
0807 18:19:58.733614 (+ 1951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:58.733650 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:58.734250 (+ 600us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:58.734757 (+ 507us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:58.736342 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:58.737714 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:58.739154 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:58.739874 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:58.740079 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:58.744762 (+ 4683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:58.744772 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:58.744794 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:19:58.744846 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:58.748320 (+ 3474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:58.749257 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:58.749266 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:58.749274 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:58.749484 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:58.750020 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:58.750024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:58.750026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:58.750919 (+ 893us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:58.751304 (+ 385us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:58.751804 (+ 500us) server_negotiation.cc:299] Negotiation successful
0807 18:19:58.752277 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":415,"threads_started":1}
I20260807 18:19:58.758404 17848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:19:58.762809 15612 raft_consensus.cc:348] T 00000000000000000000000000000000 P 902934c956df4d969031e1c3cdf270fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "902934c956df4d969031e1c3cdf270fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43239 } }
I20260807 18:19:58.763311 15612 raft_consensus.cc:374] T 00000000000000000000000000000000 P 902934c956df4d969031e1c3cdf270fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:19:58.763447 15612 raft_consensus.cc:729] T 00000000000000000000000000000000 P 902934c956df4d969031e1c3cdf270fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 902934c956df4d969031e1c3cdf270fc, State: Initialized, Role: FOLLOWER
I20260807 18:19:58.764241 15612 consensus_queue.cc:260] T 00000000000000000000000000000000 P 902934c956df4d969031e1c3cdf270fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "902934c956df4d969031e1c3cdf270fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43239 } }
I20260807 18:19:58.764585 15612 raft_consensus.cc:388] T 00000000000000000000000000000000 P 902934c956df4d969031e1c3cdf270fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:19:58.764779 15612 raft_consensus.cc:482] T 00000000000000000000000000000000 P 902934c956df4d969031e1c3cdf270fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:19:58.765007 15612 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 902934c956df4d969031e1c3cdf270fc [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:19:58.768199 15612 raft_consensus.cc:504] T 00000000000000000000000000000000 P 902934c956df4d969031e1c3cdf270fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "902934c956df4d969031e1c3cdf270fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43239 } }
I20260807 18:19:58.768874 15612 leader_election.cc:302] T 00000000000000000000000000000000 P 902934c956df4d969031e1c3cdf270fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 902934c956df4d969031e1c3cdf270fc; no voters:
I20260807 18:19:58.769618 15612 leader_election.cc:288] T 00000000000000000000000000000000 P 902934c956df4d969031e1c3cdf270fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:19:58.769770 15631 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 902934c956df4d969031e1c3cdf270fc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:19:58.771140 15631 raft_consensus.cc:686] T 00000000000000000000000000000000 P 902934c956df4d969031e1c3cdf270fc [term 1 LEADER]: Becoming Leader. State: Replica: 902934c956df4d969031e1c3cdf270fc, State: Running, Role: LEADER
I20260807 18:19:58.771723 15631 consensus_queue.cc:237] T 00000000000000000000000000000000 P 902934c956df4d969031e1c3cdf270fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "902934c956df4d969031e1c3cdf270fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43239 } }
I20260807 18:19:58.772426 15612 sys_catalog.cc:564] T 00000000000000000000000000000000 P 902934c956df4d969031e1c3cdf270fc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:19:58.773909 15635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 902934c956df4d969031e1c3cdf270fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 902934c956df4d969031e1c3cdf270fc. Latest consensus state: current_term: 1 leader_uuid: "902934c956df4d969031e1c3cdf270fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "902934c956df4d969031e1c3cdf270fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43239 } } }
I20260807 18:19:58.774590 15635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 902934c956df4d969031e1c3cdf270fc [sys.catalog]: This master's current role is: LEADER
I20260807 18:19:58.774410 15633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 902934c956df4d969031e1c3cdf270fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "902934c956df4d969031e1c3cdf270fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "902934c956df4d969031e1c3cdf270fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43239 } } }
I20260807 18:19:58.774909 15633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 902934c956df4d969031e1c3cdf270fc [sys.catalog]: This master's current role is: LEADER
I20260807 18:19:58.789630 15617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:19:58.763437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36289 (local address 127.0.0.1:43239)
0807 18:19:58.763682 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 18:19:58.763693 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:19:58.764171 (+ 478us) server_negotiation.cc:407] Connection header received
0807 18:19:58.764419 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:19:58.764424 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:19:58.764492 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:19:58.764664 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:19:58.766675 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:58.767605 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:58.769582 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:19:58.770371 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:19:58.770621 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:19:58.773970 (+ 3349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:19:58.773980 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:19:58.773988 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:19:58.774058 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:19:58.784896 (+ 10838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:58.786249 (+ 1353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:58.786258 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:58.786264 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:58.786622 (+ 358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:19:58.787271 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:19:58.787279 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:19:58.787285 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:19:58.787832 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:19:58.788241 (+ 409us) server_negotiation.cc:1035] Waiting for connection context
0807 18:19:58.788704 (+ 463us) server_negotiation.cc:299] Negotiation successful
0807 18:19:58.789062 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 18:19:58.791823 15639 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:19:58.797433 15639 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:19:58.807755 15639 catalog_manager.cc:1347] Generated new cluster ID: c409e2f562cc4cf79202511f7719b577
I20260807 18:19:58.807902 15639 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:19:58.845521 15639 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:19:58.847203 15639 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:19:58.854411 15639 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 902934c956df4d969031e1c3cdf270fc: Generated new TSK 0
I20260807 18:19:58.855468 15639 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:19:58.863351 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 15519
I20260807 18:19:58.864559 15607 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:19:58.961699 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 15519
2026-08-07T18:19:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2442 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:19:59 dist-test-slave-39l8 krb5kdc[15660](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:19:59 dist-test-slave-39l8 krb5kdc[15660](info): set up 2 sockets
Aug 07 18:19:59 dist-test-slave-39l8 krb5kdc[15660](info): commencing operation
krb5kdc: starting...
W20260807 18:20:00.352846 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.343s user 0.006s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:20:00 dist-test-slave-39l8 krb5kdc[15660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:20:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:20:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:20:00.732558 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34645
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34645
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:20:01.126616 15676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:20:01.127125 15676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:20:01.127301 15676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:20:01.135037 15676 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:20:01.135236 15676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:20:01.135342 15676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:20:01.135466 15676 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:20:01.135548 15676 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:20:01.146296 15676 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34645
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:20:01.147817 15676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:20:01.149272 15676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:20:01.162801 15683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:01.162810 15689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:01.163575 15684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:20:01.164134 15676 server_base.cc:1034] running on GCE node
I20260807 18:20:01.165199 15676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:20:01.167964 15676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:20:01.169345 15676 hybrid_clock.cc:648] HybridClock initialized: now 1786126801169272 us; error 63 us; skew 500 ppm
I20260807 18:20:01.169842 15676 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:20:01 dist-test-slave-39l8 krb5kdc[15660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126801, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:20:01.174016 15676 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:20:01.176344 15676 webserver.cc:460] Webserver started at http://127.0.0.1:45331/ using document root <none> and password file <none>
I20260807 18:20:01.177165 15676 fs_manager.cc:359] Metadata directory not provided
I20260807 18:20:01.177331 15676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:20:01.177680 15676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:20:01.181635 15676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "f9745a767b3344abab5339cf4b951269"
format_stamp: "Formatted at 2026-08-07 18:20:01 on dist-test-slave-39l8"
server_key: "160627adcf1b293113f3a484eaae4bf7"
server_key_iv: "9a6eaacda1f72d2bb3ba19ddefc997d5"
server_key_version: "encryptionkey@0"
I20260807 18:20:01.183030 15676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "f9745a767b3344abab5339cf4b951269"
format_stamp: "Formatted at 2026-08-07 18:20:01 on dist-test-slave-39l8"
server_key: "160627adcf1b293113f3a484eaae4bf7"
server_key_iv: "9a6eaacda1f72d2bb3ba19ddefc997d5"
server_key_version: "encryptionkey@0"
I20260807 18:20:01.190404 15676 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260807 18:20:01.195715 15696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:01.197431 15676 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:20:01.197702 15676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "f9745a767b3344abab5339cf4b951269"
format_stamp: "Formatted at 2026-08-07 18:20:01 on dist-test-slave-39l8"
server_key: "160627adcf1b293113f3a484eaae4bf7"
server_key_iv: "9a6eaacda1f72d2bb3ba19ddefc997d5"
server_key_version: "encryptionkey@0"
I20260807 18:20:01.197961 15676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:20:01.224056 15676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:20:01.229209 15676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:20:01.229605 15676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:20:01.252821 15676 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34645
I20260807 18:20:01.252821 15773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34645 every 8 connection(s)
I20260807 18:20:01.254455 15676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:20:01.258781 15774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:20:01.263159 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 15676
I20260807 18:20:01.263382 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:20:01.263954 17848 external_mini_cluster.cc:1428] Setting key 3c2c0d87e531031b39d98eaec08461dd
I20260807 18:20:01.272971 15774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9745a767b3344abab5339cf4b951269: Bootstrap starting.
I20260807 18:20:01.276608 15774 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9745a767b3344abab5339cf4b951269: Neither blocks nor log segments found. Creating new log.
I20260807 18:20:01.278062 15774 log.cc:824] T 00000000000000000000000000000000 P f9745a767b3344abab5339cf4b951269: Log is configured to *not* fsync() on all Append() calls
I20260807 18:20:01.278757 15777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:01.266260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38793 (local address 127.0.0.1:34645)
0807 18:20:01.267240 (+ 980us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:01.267252 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:01.267297 (+ 45us) server_negotiation.cc:407] Connection header received
0807 18:20:01.269123 (+ 1826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:01.269153 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:01.269647 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:01.270124 (+ 477us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:01.270832 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:01.272202 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:01.273649 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:01.274384 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:01.274620 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:01.275944 (+ 1324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:01.275952 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:01.275969 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:20:01.276615 (+ 646us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:01.276717 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:20:01.276745 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:01.276894 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:01.277213 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 18:20:01.277538 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":475,"threads_started":1}
I20260807 18:20:01.281440 15774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9745a767b3344abab5339cf4b951269: No bootstrap required, opened a new log
I20260807 18:20:01.283182 17848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:20:01.292701 15774 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9745a767b3344abab5339cf4b951269 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9745a767b3344abab5339cf4b951269" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34645 } }
I20260807 18:20:01.293150 15774 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9745a767b3344abab5339cf4b951269 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:20:01.293287 15774 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9745a767b3344abab5339cf4b951269 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9745a767b3344abab5339cf4b951269, State: Initialized, Role: FOLLOWER
I20260807 18:20:01.294060 15774 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9745a767b3344abab5339cf4b951269 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9745a767b3344abab5339cf4b951269" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34645 } }
I20260807 18:20:01.294394 15774 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9745a767b3344abab5339cf4b951269 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:20:01.294560 15774 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9745a767b3344abab5339cf4b951269 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:20:01.294842 15774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9745a767b3344abab5339cf4b951269 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:20:01.296890 15777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:01.289301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48603 (local address 127.0.0.1:34645)
0807 18:20:01.289534 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:01.289541 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:01.290086 (+ 545us) server_negotiation.cc:407] Connection header received
0807 18:20:01.290290 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:01.290298 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:01.290383 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:01.290514 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:01.291175 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:01.292243 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:01.293810 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:01.294501 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:01.294657 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:01.296021 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:01.296028 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:01.296033 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:20:01.296060 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:01.296113 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:20:01.296134 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:01.296248 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:01.296423 (+ 175us) server_negotiation.cc:299] Negotiation successful
0807 18:20:01.296536 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 18:20:01.298019 15774 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9745a767b3344abab5339cf4b951269 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9745a767b3344abab5339cf4b951269" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34645 } }
I20260807 18:20:01.298635 15774 leader_election.cc:302] T 00000000000000000000000000000000 P f9745a767b3344abab5339cf4b951269 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9745a767b3344abab5339cf4b951269; no voters:
I20260807 18:20:01.299368 15774 leader_election.cc:288] T 00000000000000000000000000000000 P f9745a767b3344abab5339cf4b951269 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:20:01.299538 15786 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9745a767b3344abab5339cf4b951269 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:20:01.301918 15774 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9745a767b3344abab5339cf4b951269 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:20:01.305035 15786 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9745a767b3344abab5339cf4b951269 [term 1 LEADER]: Becoming Leader. State: Replica: f9745a767b3344abab5339cf4b951269, State: Running, Role: LEADER
I20260807 18:20:01.305787 15786 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9745a767b3344abab5339cf4b951269 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9745a767b3344abab5339cf4b951269" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34645 } }
I20260807 18:20:01.309834 15794 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9745a767b3344abab5339cf4b951269 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9745a767b3344abab5339cf4b951269. Latest consensus state: current_term: 1 leader_uuid: "f9745a767b3344abab5339cf4b951269" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9745a767b3344abab5339cf4b951269" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34645 } } }
I20260807 18:20:01.309688 15793 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9745a767b3344abab5339cf4b951269 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9745a767b3344abab5339cf4b951269" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9745a767b3344abab5339cf4b951269" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34645 } } }
I20260807 18:20:01.310262 15793 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9745a767b3344abab5339cf4b951269 [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:01.310264 15794 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9745a767b3344abab5339cf4b951269 [sys.catalog]: This master's current role is: LEADER
W20260807 18:20:01.315616 15811 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f9745a767b3344abab5339cf4b951269: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:20:01.315773 15811 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:20:01.315945 15812 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:20:01.317672 15812 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:20:01.327993 15812 catalog_manager.cc:1347] Generated new cluster ID: bdfd134e92ac4e62a9951f2b0df159ea
I20260807 18:20:01.328166 15812 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:20:01.353188 15812 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:20:01.354656 15812 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:20:01.363389 15812 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9745a767b3344abab5339cf4b951269: Generated new TSK 0
I20260807 18:20:01.364352 15812 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:20:01.378399 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 15676
W20260807 18:20:01.379973 15768 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:38793
W20260807 18:20:01.380960 17848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 15676: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:20:01.381155 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 15676
2026-08-07T18:20:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2416 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:20:01 dist-test-slave-39l8 krb5kdc[15820](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:20:01 dist-test-slave-39l8 krb5kdc[15820](info): set up 2 sockets
Aug 07 18:20:01 dist-test-slave-39l8 krb5kdc[15820](info): commencing operation
krb5kdc: starting...
W20260807 18:20:02.746935 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.320s user 0.002s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:20:02 dist-test-slave-39l8 krb5kdc[15820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:20:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:20:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:20:03.110041 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45449
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45449
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:20:03.492651 15842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:20:03.493077 15842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:20:03.493252 15842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:20:03.500757 15842 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:20:03.500946 15842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:20:03.501071 15842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:20:03.501188 15842 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:20:03.501279 15842 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:20:03.511803 15842 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:20:03.513399 15842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:20:03.514818 15842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:20:03.527274 15849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:03.527467 15852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:03.527503 15850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:20:03.528551 15842 server_base.cc:1034] running on GCE node
I20260807 18:20:03.529423 15842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:20:03.531186 15842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:20:03.532456 15842 hybrid_clock.cc:648] HybridClock initialized: now 1786126803532416 us; error 35 us; skew 500 ppm
I20260807 18:20:03.532873 15842 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:20:03 dist-test-slave-39l8 krb5kdc[15820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126803, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:20:03.536644 15842 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:20:03.538489 15842 webserver.cc:460] Webserver started at http://127.0.0.1:41303/ using document root <none> and password file <none>
I20260807 18:20:03.539393 15842 fs_manager.cc:359] Metadata directory not provided
I20260807 18:20:03.539541 15842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:20:03.539901 15842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:20:03.543275 15842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "dcfc336c9f024d8a84461a2d6c436e98"
format_stamp: "Formatted at 2026-08-07 18:20:03 on dist-test-slave-39l8"
server_key: "17182fd2ab455fa8426c70a724b47f68"
server_key_iv: "aef18a02505a188c559dfaf92e672503"
server_key_version: "encryptionkey@0"
I20260807 18:20:03.544229 15842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "dcfc336c9f024d8a84461a2d6c436e98"
format_stamp: "Formatted at 2026-08-07 18:20:03 on dist-test-slave-39l8"
server_key: "17182fd2ab455fa8426c70a724b47f68"
server_key_iv: "aef18a02505a188c559dfaf92e672503"
server_key_version: "encryptionkey@0"
I20260807 18:20:03.550062 15842 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260807 18:20:03.555382 15865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:03.557199 15842 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:20:03.557451 15842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "dcfc336c9f024d8a84461a2d6c436e98"
format_stamp: "Formatted at 2026-08-07 18:20:03 on dist-test-slave-39l8"
server_key: "17182fd2ab455fa8426c70a724b47f68"
server_key_iv: "aef18a02505a188c559dfaf92e672503"
server_key_version: "encryptionkey@0"
I20260807 18:20:03.557716 15842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:20:03.564401 15842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:20:03.569494 15842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:20:03.569947 15842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:20:03.593027 15842 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45449
I20260807 18:20:03.593108 15930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45449 every 8 connection(s)
I20260807 18:20:03.594928 15842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:20:03.596782 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 15842
I20260807 18:20:03.597004 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:20:03.597540 17848 external_mini_cluster.cc:1428] Setting key 3d3205f8816f758268465a8d0e9e5542
I20260807 18:20:03.600172 15931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:20:03.607352 15936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:03.599717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34053 (local address 127.0.0.1:45449)
0807 18:20:03.600767 (+ 1050us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:03.600781 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:03.600832 (+ 51us) server_negotiation.cc:407] Connection header received
0807 18:20:03.602935 (+ 2103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:03.602964 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:03.603417 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:03.603932 (+ 515us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:03.605264 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:03.605272 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:03.605288 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:20:03.605305 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:03.605389 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:20:03.605414 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:03.605724 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:03.605903 (+ 179us) server_negotiation.cc:299] Negotiation successful
0807 18:20:03.606147 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":783,"thread_start_us":448,"threads_started":1}
I20260807 18:20:03.612016 17848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:20:03.613577 15931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcfc336c9f024d8a84461a2d6c436e98: Bootstrap starting.
I20260807 18:20:03.617524 15931 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dcfc336c9f024d8a84461a2d6c436e98: Neither blocks nor log segments found. Creating new log.
I20260807 18:20:03.618963 15931 log.cc:824] T 00000000000000000000000000000000 P dcfc336c9f024d8a84461a2d6c436e98: Log is configured to *not* fsync() on all Append() calls
I20260807 18:20:03.620036 15936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:03.616851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60365 (local address 127.0.0.1:45449)
0807 18:20:03.617025 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:03.617030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:03.617526 (+ 496us) server_negotiation.cc:407] Connection header received
0807 18:20:03.617761 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:03.617765 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:03.617825 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:03.617955 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:03.619077 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:03.619081 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:03.619085 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:20:03.619105 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:03.619176 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:20:03.619200 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:03.619342 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:03.619479 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 18:20:03.619551 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:20:03.622641 15931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcfc336c9f024d8a84461a2d6c436e98: No bootstrap required, opened a new log
I20260807 18:20:03.631215 15931 raft_consensus.cc:348] T 00000000000000000000000000000000 P dcfc336c9f024d8a84461a2d6c436e98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcfc336c9f024d8a84461a2d6c436e98" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45449 } }
I20260807 18:20:03.631642 15931 raft_consensus.cc:374] T 00000000000000000000000000000000 P dcfc336c9f024d8a84461a2d6c436e98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:20:03.631767 15931 raft_consensus.cc:729] T 00000000000000000000000000000000 P dcfc336c9f024d8a84461a2d6c436e98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcfc336c9f024d8a84461a2d6c436e98, State: Initialized, Role: FOLLOWER
I20260807 18:20:03.632427 15931 consensus_queue.cc:260] T 00000000000000000000000000000000 P dcfc336c9f024d8a84461a2d6c436e98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcfc336c9f024d8a84461a2d6c436e98" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45449 } }
I20260807 18:20:03.632706 15931 raft_consensus.cc:388] T 00000000000000000000000000000000 P dcfc336c9f024d8a84461a2d6c436e98 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:20:03.632891 15931 raft_consensus.cc:482] T 00000000000000000000000000000000 P dcfc336c9f024d8a84461a2d6c436e98 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:20:03.633175 15931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dcfc336c9f024d8a84461a2d6c436e98 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:20:03.636034 15931 raft_consensus.cc:504] T 00000000000000000000000000000000 P dcfc336c9f024d8a84461a2d6c436e98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcfc336c9f024d8a84461a2d6c436e98" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45449 } }
I20260807 18:20:03.636665 15931 leader_election.cc:302] T 00000000000000000000000000000000 P dcfc336c9f024d8a84461a2d6c436e98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dcfc336c9f024d8a84461a2d6c436e98; no voters:
I20260807 18:20:03.637415 15931 leader_election.cc:288] T 00000000000000000000000000000000 P dcfc336c9f024d8a84461a2d6c436e98 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:20:03.637569 15944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dcfc336c9f024d8a84461a2d6c436e98 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:20:03.638955 15944 raft_consensus.cc:686] T 00000000000000000000000000000000 P dcfc336c9f024d8a84461a2d6c436e98 [term 1 LEADER]: Becoming Leader. State: Replica: dcfc336c9f024d8a84461a2d6c436e98, State: Running, Role: LEADER
I20260807 18:20:03.639600 15944 consensus_queue.cc:237] T 00000000000000000000000000000000 P dcfc336c9f024d8a84461a2d6c436e98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcfc336c9f024d8a84461a2d6c436e98" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45449 } }
I20260807 18:20:03.640198 15931 sys_catalog.cc:564] T 00000000000000000000000000000000 P dcfc336c9f024d8a84461a2d6c436e98 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:20:03.643216 15946 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcfc336c9f024d8a84461a2d6c436e98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dcfc336c9f024d8a84461a2d6c436e98. Latest consensus state: current_term: 1 leader_uuid: "dcfc336c9f024d8a84461a2d6c436e98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcfc336c9f024d8a84461a2d6c436e98" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45449 } } }
I20260807 18:20:03.643786 15946 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcfc336c9f024d8a84461a2d6c436e98 [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:03.643216 15945 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcfc336c9f024d8a84461a2d6c436e98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dcfc336c9f024d8a84461a2d6c436e98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcfc336c9f024d8a84461a2d6c436e98" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45449 } } }
I20260807 18:20:03.644096 15945 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcfc336c9f024d8a84461a2d6c436e98 [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:03.646016 15955 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:20:03.651330 15955 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:20:03.661628 15955 catalog_manager.cc:1347] Generated new cluster ID: 1a7810b439764b2f8eb1def64ce1ef3c
I20260807 18:20:03.661772 15955 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:20:03.683436 15955 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:20:03.683983 15955 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:20:03.692056 15955 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dcfc336c9f024d8a84461a2d6c436e98: Generated new TSK 0
I20260807 18:20:03.693403 15955 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:20:03.707402 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 15842
W20260807 18:20:03.709281 15923 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34053
W20260807 18:20:03.710220 17848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 15842: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:20:03.710405 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 15842
2026-08-07T18:20:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2328 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:20:03 dist-test-slave-39l8 krb5kdc[15977](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:20:03 dist-test-slave-39l8 krb5kdc[15977](info): set up 2 sockets
Aug 07 18:20:03 dist-test-slave-39l8 krb5kdc[15977](info): commencing operation
krb5kdc: starting...
W20260807 18:20:05.076249 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.319s user 0.000s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:20:05 dist-test-slave-39l8 krb5kdc[15977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:20:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:20:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:20:05.440207 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38053
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:20:05.815555 15992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:20:05.815881 15992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:20:05.816036 15992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:20:05.823763 15992 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:20:05.823899 15992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:20:05.823987 15992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:20:05.824093 15992 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:20:05.824182 15992 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:20:05.834971 15992 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:20:05.836472 15992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:20:05.837816 15992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:20:05.850543 15997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:05.850616 15998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:05.850651 16000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:20:05.852519 15992 server_base.cc:1034] running on GCE node
I20260807 18:20:05.853471 15992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:20:05.855206 15992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:20:05.856431 15992 hybrid_clock.cc:648] HybridClock initialized: now 1786126805856378 us; error 47 us; skew 500 ppm
I20260807 18:20:05.856827 15992 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:20:05 dist-test-slave-39l8 krb5kdc[15977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126805, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:20:05.865764 15992 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:20:05.867805 15992 webserver.cc:460] Webserver started at http://127.0.0.1:34467/ using document root <none> and password file <none>
I20260807 18:20:05.868660 15992 fs_manager.cc:359] Metadata directory not provided
I20260807 18:20:05.868839 15992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:20:05.869202 15992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:20:05.872668 15992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "15a014a0fe3046bd812c602614d88b2c"
format_stamp: "Formatted at 2026-08-07 18:20:05 on dist-test-slave-39l8"
server_key: "349957aa06e371094142f6f26425a4c6"
server_key_iv: "48acd54092c7ca6369aef9a1a18b9f71"
server_key_version: "encryptionkey@0"
I20260807 18:20:05.873549 15992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "15a014a0fe3046bd812c602614d88b2c"
format_stamp: "Formatted at 2026-08-07 18:20:05 on dist-test-slave-39l8"
server_key: "349957aa06e371094142f6f26425a4c6"
server_key_iv: "48acd54092c7ca6369aef9a1a18b9f71"
server_key_version: "encryptionkey@0"
I20260807 18:20:05.879799 15992 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260807 18:20:05.884106 16012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:05.885720 15992 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260807 18:20:05.885960 15992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "15a014a0fe3046bd812c602614d88b2c"
format_stamp: "Formatted at 2026-08-07 18:20:05 on dist-test-slave-39l8"
server_key: "349957aa06e371094142f6f26425a4c6"
server_key_iv: "48acd54092c7ca6369aef9a1a18b9f71"
server_key_version: "encryptionkey@0"
I20260807 18:20:05.886197 15992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:20:05.909175 15992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:20:05.914184 15992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:20:05.914536 15992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:20:05.936007 15992 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38053
I20260807 18:20:05.936054 16089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38053 every 8 connection(s)
I20260807 18:20:05.937505 15992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:20:05.938015 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 15992
I20260807 18:20:05.938310 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:20:05.939056 17848 external_mini_cluster.cc:1428] Setting key 1eb37d802cc95b236b68dcd84e0f8eec
I20260807 18:20:05.942339 16090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:20:05.956806 16090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15a014a0fe3046bd812c602614d88b2c: Bootstrap starting.
Aug 07 18:20:05 dist-test-slave-39l8 krb5kdc[15977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 18:20:05.961280 16090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15a014a0fe3046bd812c602614d88b2c: Neither blocks nor log segments found. Creating new log.
I20260807 18:20:05.962896 16090 log.cc:824] T 00000000000000000000000000000000 P 15a014a0fe3046bd812c602614d88b2c: Log is configured to *not* fsync() on all Append() calls
I20260807 18:20:05.967135 16090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15a014a0fe3046bd812c602614d88b2c: No bootstrap required, opened a new log
I20260807 18:20:05.969165 16094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:05.941266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:44911 (local address 127.0.0.1:38053)
0807 18:20:05.942397 (+ 1131us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:05.942423 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:05.943159 (+ 736us) server_negotiation.cc:407] Connection header received
0807 18:20:05.945565 (+ 2406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:05.945596 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:05.946196 (+ 600us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:05.946692 (+ 496us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:05.948360 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:05.950499 (+ 2139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:05.952129 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:05.953283 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:05.953497 (+ 214us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:05.958871 (+ 5374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:05.958886 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:05.958908 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:05.958987 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:05.963471 (+ 4484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:05.964858 (+ 1387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:05.964870 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:05.964879 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:05.965111 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:05.965610 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:05.965615 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:05.965619 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:05.966392 (+ 773us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:05.966759 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:05.967323 (+ 564us) server_negotiation.cc:299] Negotiation successful
0807 18:20:05.967727 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":746,"thread_start_us":438,"threads_started":1}
I20260807 18:20:05.974319 17848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:20:05.976892 16090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15a014a0fe3046bd812c602614d88b2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a014a0fe3046bd812c602614d88b2c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38053 } }
I20260807 18:20:05.977485 16090 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15a014a0fe3046bd812c602614d88b2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:20:05.977627 16090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15a014a0fe3046bd812c602614d88b2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15a014a0fe3046bd812c602614d88b2c, State: Initialized, Role: FOLLOWER
I20260807 18:20:05.978328 16090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15a014a0fe3046bd812c602614d88b2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a014a0fe3046bd812c602614d88b2c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38053 } }
I20260807 18:20:05.978588 16090 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15a014a0fe3046bd812c602614d88b2c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:20:05.978781 16090 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15a014a0fe3046bd812c602614d88b2c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:20:05.978994 16090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15a014a0fe3046bd812c602614d88b2c [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:20:05.981813 16090 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15a014a0fe3046bd812c602614d88b2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a014a0fe3046bd812c602614d88b2c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38053 } }
I20260807 18:20:05.982478 16090 leader_election.cc:302] T 00000000000000000000000000000000 P 15a014a0fe3046bd812c602614d88b2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15a014a0fe3046bd812c602614d88b2c; no voters:
I20260807 18:20:05.983290 16090 leader_election.cc:288] T 00000000000000000000000000000000 P 15a014a0fe3046bd812c602614d88b2c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:20:05.983462 16107 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15a014a0fe3046bd812c602614d88b2c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:20:05.985183 16107 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15a014a0fe3046bd812c602614d88b2c [term 1 LEADER]: Becoming Leader. State: Replica: 15a014a0fe3046bd812c602614d88b2c, State: Running, Role: LEADER
I20260807 18:20:05.985801 16090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15a014a0fe3046bd812c602614d88b2c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:20:05.986057 16107 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15a014a0fe3046bd812c602614d88b2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a014a0fe3046bd812c602614d88b2c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38053 } }
I20260807 18:20:05.992878 16108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15a014a0fe3046bd812c602614d88b2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15a014a0fe3046bd812c602614d88b2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a014a0fe3046bd812c602614d88b2c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38053 } } }
I20260807 18:20:05.993587 16108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15a014a0fe3046bd812c602614d88b2c [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:05.993552 16109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15a014a0fe3046bd812c602614d88b2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15a014a0fe3046bd812c602614d88b2c. Latest consensus state: current_term: 1 leader_uuid: "15a014a0fe3046bd812c602614d88b2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a014a0fe3046bd812c602614d88b2c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38053 } } }
I20260807 18:20:05.994416 16109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15a014a0fe3046bd812c602614d88b2c [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:06.002995 16094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:05.979636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:43707 (local address 127.0.0.1:38053)
0807 18:20:05.979842 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:05.979848 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:05.980417 (+ 569us) server_negotiation.cc:407] Connection header received
0807 18:20:05.980554 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:05.980560 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:05.980653 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:05.980787 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:05.982500 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:05.983835 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:05.985773 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:05.986502 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:05.986682 (+ 180us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:05.990599 (+ 3917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:05.990612 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:05.990621 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:05.990726 (+ 105us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:05.996636 (+ 5910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:05.998278 (+ 1642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:05.998287 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:05.998294 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:05.998582 (+ 288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:05.999509 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:05.999517 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:05.999523 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:06.000057 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:06.002048 (+ 1991us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:06.002265 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 18:20:06.002526 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 18:20:06.003377 16122 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:20:06.009490 16122 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:20:06.021724 16122 catalog_manager.cc:1347] Generated new cluster ID: 5155c67ff95749999432f41f1f6ae057
I20260807 18:20:06.021862 16122 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:20:06.037990 16122 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:20:06.039542 16122 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:20:06.057385 16122 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15a014a0fe3046bd812c602614d88b2c: Generated new TSK 0
I20260807 18:20:06.058537 16122 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:20:06.074767 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 15992
I20260807 18:20:06.076541 16085 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:20:06.171741 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 15992
W20260807 18:20:06.199203 15971 connection.cc:535] client connection to 127.0.0.1:38053 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38053): Connection reset by peer (error 104)
2026-08-07T18:20:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2462 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:20:06 dist-test-slave-39l8 krb5kdc[16138](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:20:06 dist-test-slave-39l8 krb5kdc[16138](info): set up 2 sockets
Aug 07 18:20:06 dist-test-slave-39l8 krb5kdc[16138](info): commencing operation
krb5kdc: starting...
W20260807 18:20:07.567549 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.348s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:20:07 dist-test-slave-39l8 krb5kdc[16138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126807, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:20:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:20:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:20:07.935055 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35899
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:20:08.330591 16157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:20:08.331177 16157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:20:08.331385 16157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:20:08.339613 16157 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:20:08.339816 16157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:20:08.339932 16157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:20:08.340068 16157 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:20:08.340149 16157 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:20:08.351092 16157 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:20:08.352695 16157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:20:08.354275 16157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:20:08.369367 16169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:08.369439 16168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:08.369414 16172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:20:08.370604 16157 server_base.cc:1034] running on GCE node
I20260807 18:20:08.371729 16157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:20:08.373497 16157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:20:08.374836 16157 hybrid_clock.cc:648] HybridClock initialized: now 1786126808374763 us; error 81 us; skew 500 ppm
I20260807 18:20:08.375334 16157 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:20:08 dist-test-slave-39l8 krb5kdc[16138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126808, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:20:08.379698 16157 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:20:08.382687 16157 webserver.cc:460] Webserver started at http://127.0.0.1:40745/ using document root <none> and password file <none>
I20260807 18:20:08.383780 16157 fs_manager.cc:359] Metadata directory not provided
I20260807 18:20:08.383973 16157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:20:08.384349 16157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:20:08.388001 16157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "ff756e45e1dd4240abcc1b314894a77a"
format_stamp: "Formatted at 2026-08-07 18:20:08 on dist-test-slave-39l8"
server_key: "daf47090b12b8f737eebcc00e9a10117"
server_key_iv: "d27e4d028e475afd392c0a13de7af5ac"
server_key_version: "encryptionkey@0"
I20260807 18:20:08.388960 16157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "ff756e45e1dd4240abcc1b314894a77a"
format_stamp: "Formatted at 2026-08-07 18:20:08 on dist-test-slave-39l8"
server_key: "daf47090b12b8f737eebcc00e9a10117"
server_key_iv: "d27e4d028e475afd392c0a13de7af5ac"
server_key_version: "encryptionkey@0"
I20260807 18:20:08.395830 16157 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 18:20:08.400782 16183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:08.402464 16157 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:20:08.402806 16157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "ff756e45e1dd4240abcc1b314894a77a"
format_stamp: "Formatted at 2026-08-07 18:20:08 on dist-test-slave-39l8"
server_key: "daf47090b12b8f737eebcc00e9a10117"
server_key_iv: "d27e4d028e475afd392c0a13de7af5ac"
server_key_version: "encryptionkey@0"
I20260807 18:20:08.403071 16157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:20:08.429023 16157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:20:08.435284 16157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:20:08.435654 16157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:20:08.459026 16157 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35899
I20260807 18:20:08.459069 16257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35899 every 8 connection(s)
I20260807 18:20:08.460628 16157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:20:08.464653 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 16157
I20260807 18:20:08.464877 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:20:08.465517 17848 external_mini_cluster.cc:1428] Setting key f0de5aba9b01a55954c1e62ac38b2b3d
I20260807 18:20:08.466555 16259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:20:08.480520 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:08.467796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:40221 (local address 127.0.0.1:35899)
0807 18:20:08.468753 (+ 957us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:08.468766 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:08.468822 (+ 56us) server_negotiation.cc:407] Connection header received
0807 18:20:08.470609 (+ 1787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:08.470637 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:08.471211 (+ 574us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:08.471687 (+ 476us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:08.472336 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:08.473810 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:08.475367 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:08.476142 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:08.476378 (+ 236us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:08.477861 (+ 1483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:08.477870 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:08.477891 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:20:08.478536 (+ 645us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:08.478630 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:20:08.478666 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:08.478927 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:08.479229 (+ 302us) server_negotiation.cc:299] Negotiation successful
0807 18:20:08.479440 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":685,"thread_start_us":428,"threads_started":1}
I20260807 18:20:08.482040 16259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff756e45e1dd4240abcc1b314894a77a: Bootstrap starting.
I20260807 18:20:08.485836 17848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:20:08.486367 16259 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff756e45e1dd4240abcc1b314894a77a: Neither blocks nor log segments found. Creating new log.
I20260807 18:20:08.487672 16259 log.cc:824] T 00000000000000000000000000000000 P ff756e45e1dd4240abcc1b314894a77a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:20:08.491833 16259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff756e45e1dd4240abcc1b314894a77a: No bootstrap required, opened a new log
I20260807 18:20:08.499897 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:08.491160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:59875 (local address 127.0.0.1:35899)
0807 18:20:08.491396 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:08.491404 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:08.491781 (+ 377us) server_negotiation.cc:407] Connection header received
0807 18:20:08.492101 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:08.492109 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:08.492184 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:08.492291 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:08.493111 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:08.494005 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:08.496368 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:08.496925 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:08.497057 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:08.498672 (+ 1615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:08.498678 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:08.498681 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:20:08.498770 (+ 89us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:08.498857 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:20:08.498878 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:08.498989 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:08.499493 (+ 504us) server_negotiation.cc:299] Negotiation successful
0807 18:20:08.499564 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260807 18:20:08.503485 16259 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff756e45e1dd4240abcc1b314894a77a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff756e45e1dd4240abcc1b314894a77a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35899 } }
I20260807 18:20:08.503964 16259 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff756e45e1dd4240abcc1b314894a77a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:20:08.504133 16259 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff756e45e1dd4240abcc1b314894a77a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff756e45e1dd4240abcc1b314894a77a, State: Initialized, Role: FOLLOWER
I20260807 18:20:08.504971 16259 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff756e45e1dd4240abcc1b314894a77a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff756e45e1dd4240abcc1b314894a77a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35899 } }
I20260807 18:20:08.505267 16259 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff756e45e1dd4240abcc1b314894a77a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:20:08.505487 16259 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff756e45e1dd4240abcc1b314894a77a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:20:08.505709 16259 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff756e45e1dd4240abcc1b314894a77a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:20:08.508986 16259 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff756e45e1dd4240abcc1b314894a77a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff756e45e1dd4240abcc1b314894a77a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35899 } }
I20260807 18:20:08.509724 16259 leader_election.cc:302] T 00000000000000000000000000000000 P ff756e45e1dd4240abcc1b314894a77a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff756e45e1dd4240abcc1b314894a77a; no voters:
I20260807 18:20:08.510499 16259 leader_election.cc:288] T 00000000000000000000000000000000 P ff756e45e1dd4240abcc1b314894a77a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:20:08.510673 16274 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff756e45e1dd4240abcc1b314894a77a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:20:08.511633 16274 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff756e45e1dd4240abcc1b314894a77a [term 1 LEADER]: Becoming Leader. State: Replica: ff756e45e1dd4240abcc1b314894a77a, State: Running, Role: LEADER
I20260807 18:20:08.512370 16274 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff756e45e1dd4240abcc1b314894a77a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff756e45e1dd4240abcc1b314894a77a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35899 } }
I20260807 18:20:08.513325 16259 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff756e45e1dd4240abcc1b314894a77a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:20:08.516536 16277 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff756e45e1dd4240abcc1b314894a77a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff756e45e1dd4240abcc1b314894a77a. Latest consensus state: current_term: 1 leader_uuid: "ff756e45e1dd4240abcc1b314894a77a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff756e45e1dd4240abcc1b314894a77a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35899 } } }
I20260807 18:20:08.516571 16275 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff756e45e1dd4240abcc1b314894a77a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff756e45e1dd4240abcc1b314894a77a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff756e45e1dd4240abcc1b314894a77a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35899 } } }
I20260807 18:20:08.517156 16277 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff756e45e1dd4240abcc1b314894a77a [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:08.517239 16275 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff756e45e1dd4240abcc1b314894a77a [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:08.520216 16282 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:20:08.526017 16282 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:20:08.527266 16302 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 18:20:08.536087 16282 catalog_manager.cc:1347] Generated new cluster ID: bf83d8ac572b4cc78b414b3fcf806676
I20260807 18:20:08.536237 16282 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:20:08.549973 16282 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:20:08.552099 16282 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:20:08.568877 16282 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff756e45e1dd4240abcc1b314894a77a: Generated new TSK 0
I20260807 18:20:08.570338 16282 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:20:08.587435 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 16157
W20260807 18:20:08.589218 16252 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.38:40221
W20260807 18:20:08.590337 17848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 16157: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:20:08.590539 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 16157
W20260807 18:20:08.619112 16135 connection.cc:535] client connection to 127.0.0.1:35899 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35899): Connection reset by peer (error 104)
2026-08-07T18:20:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2420 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:20:08 dist-test-slave-39l8 krb5kdc[16306](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:20:08 dist-test-slave-39l8 krb5kdc[16306](info): set up 2 sockets
Aug 07 18:20:08 dist-test-slave-39l8 krb5kdc[16306](info): commencing operation
krb5kdc: starting...
W20260807 18:20:09.987491 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.348s user 0.004s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:20:10 dist-test-slave-39l8 krb5kdc[16306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:20:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:20:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:20:10.364320 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37625
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:20:10.745579 16331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:20:10.745967 16331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:20:10.746212 16331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:20:10.753787 16331 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:20:10.753985 16331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:20:10.754101 16331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:20:10.754217 16331 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:20:10.754297 16331 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:20:10.764671 16331 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:20:10.766090 16331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:20:10.767643 16331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:20:10.780902 16337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:10.780802 16338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:10.780829 16340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:20:10.781857 16331 server_base.cc:1034] running on GCE node
I20260807 18:20:10.782816 16331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:20:10.784415 16331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:20:10.785655 16331 hybrid_clock.cc:648] HybridClock initialized: now 1786126810785601 us; error 85 us; skew 500 ppm
I20260807 18:20:10.786044 16331 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:20:10 dist-test-slave-39l8 krb5kdc[16306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126810, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:20:10.789623 16331 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:20:10.791491 16331 webserver.cc:460] Webserver started at http://127.0.0.1:42369/ using document root <none> and password file <none>
I20260807 18:20:10.792243 16331 fs_manager.cc:359] Metadata directory not provided
I20260807 18:20:10.792356 16331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:20:10.792644 16331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:20:10.795953 16331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "51c1fab3e4c24630a990ab8344349228"
format_stamp: "Formatted at 2026-08-07 18:20:10 on dist-test-slave-39l8"
server_key: "32075c347c4d67886fb6adea3dc68178"
server_key_iv: "67997455edc76e6396e2701bcbde4282"
server_key_version: "encryptionkey@0"
I20260807 18:20:10.796769 16331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "51c1fab3e4c24630a990ab8344349228"
format_stamp: "Formatted at 2026-08-07 18:20:10 on dist-test-slave-39l8"
server_key: "32075c347c4d67886fb6adea3dc68178"
server_key_iv: "67997455edc76e6396e2701bcbde4282"
server_key_version: "encryptionkey@0"
I20260807 18:20:10.802678 16331 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 18:20:10.806818 16349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:10.808429 16331 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260807 18:20:10.808676 16331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "51c1fab3e4c24630a990ab8344349228"
format_stamp: "Formatted at 2026-08-07 18:20:10 on dist-test-slave-39l8"
server_key: "32075c347c4d67886fb6adea3dc68178"
server_key_iv: "67997455edc76e6396e2701bcbde4282"
server_key_version: "encryptionkey@0"
I20260807 18:20:10.808905 16331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:20:10.815680 16331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:20:10.820577 16331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:20:10.820931 16331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:20:10.841895 16331 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37625
I20260807 18:20:10.841948 16418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37625 every 8 connection(s)
I20260807 18:20:10.843595 16331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:20:10.847947 16419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:20:10.851541 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 16331
I20260807 18:20:10.851743 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:20:10.852314 17848 external_mini_cluster.cc:1428] Setting key 182d761e56674da2459c87c017ecab52
I20260807 18:20:10.861035 16424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:10.854518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:44145 (local address 127.0.0.1:37625)
0807 18:20:10.855425 (+ 907us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:10.855439 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:10.855484 (+ 45us) server_negotiation.cc:407] Connection header received
0807 18:20:10.857242 (+ 1758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:10.857267 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:10.857666 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:10.858137 (+ 471us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:10.859358 (+ 1221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:10.859365 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:10.859380 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:20:10.859399 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:10.859460 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:20:10.859485 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:10.859729 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:10.859871 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 18:20:10.860076 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":349,"threads_started":1}
I20260807 18:20:10.861441 16419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51c1fab3e4c24630a990ab8344349228: Bootstrap starting.
I20260807 18:20:10.865033 17848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:20:10.865037 16419 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51c1fab3e4c24630a990ab8344349228: Neither blocks nor log segments found. Creating new log.
I20260807 18:20:10.866139 16419 log.cc:824] T 00000000000000000000000000000000 P 51c1fab3e4c24630a990ab8344349228: Log is configured to *not* fsync() on all Append() calls
I20260807 18:20:10.869715 16419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51c1fab3e4c24630a990ab8344349228: No bootstrap required, opened a new log
I20260807 18:20:10.877789 16424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:10.870266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:60983 (local address 127.0.0.1:37625)
0807 18:20:10.870494 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:10.870504 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:10.870993 (+ 489us) server_negotiation.cc:407] Connection header received
0807 18:20:10.871225 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:10.871230 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:10.871300 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:10.871411 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:10.877005 (+ 5594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:10.877010 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:10.877013 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:20:10.877043 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:10.877109 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:20:10.877129 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:10.877302 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:10.877392 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 18:20:10.877478 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 18:20:10.878688 16419 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51c1fab3e4c24630a990ab8344349228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51c1fab3e4c24630a990ab8344349228" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37625 } }
I20260807 18:20:10.879194 16419 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51c1fab3e4c24630a990ab8344349228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:20:10.879325 16419 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51c1fab3e4c24630a990ab8344349228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51c1fab3e4c24630a990ab8344349228, State: Initialized, Role: FOLLOWER
I20260807 18:20:10.879977 16419 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51c1fab3e4c24630a990ab8344349228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51c1fab3e4c24630a990ab8344349228" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37625 } }
I20260807 18:20:10.880237 16419 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51c1fab3e4c24630a990ab8344349228 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:20:10.880347 16419 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51c1fab3e4c24630a990ab8344349228 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:20:10.880483 16419 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51c1fab3e4c24630a990ab8344349228 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:20:10.882875 16419 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51c1fab3e4c24630a990ab8344349228 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51c1fab3e4c24630a990ab8344349228" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37625 } }
I20260807 18:20:10.883430 16419 leader_election.cc:302] T 00000000000000000000000000000000 P 51c1fab3e4c24630a990ab8344349228 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51c1fab3e4c24630a990ab8344349228; no voters:
I20260807 18:20:10.884068 16419 leader_election.cc:288] T 00000000000000000000000000000000 P 51c1fab3e4c24630a990ab8344349228 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:20:10.884253 16433 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51c1fab3e4c24630a990ab8344349228 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:20:10.884986 16433 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51c1fab3e4c24630a990ab8344349228 [term 1 LEADER]: Becoming Leader. State: Replica: 51c1fab3e4c24630a990ab8344349228, State: Running, Role: LEADER
I20260807 18:20:10.885733 16433 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51c1fab3e4c24630a990ab8344349228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51c1fab3e4c24630a990ab8344349228" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37625 } }
I20260807 18:20:10.886631 16419 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51c1fab3e4c24630a990ab8344349228 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:20:10.888988 16434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51c1fab3e4c24630a990ab8344349228 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51c1fab3e4c24630a990ab8344349228" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51c1fab3e4c24630a990ab8344349228" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37625 } } }
I20260807 18:20:10.889402 16434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51c1fab3e4c24630a990ab8344349228 [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:10.890067 16435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51c1fab3e4c24630a990ab8344349228 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51c1fab3e4c24630a990ab8344349228. Latest consensus state: current_term: 1 leader_uuid: "51c1fab3e4c24630a990ab8344349228" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51c1fab3e4c24630a990ab8344349228" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37625 } } }
I20260807 18:20:10.890450 16440 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:20:10.890421 16435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51c1fab3e4c24630a990ab8344349228 [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:10.897565 16440 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:20:10.907963 16440 catalog_manager.cc:1347] Generated new cluster ID: 6cf6c6b7e4fe4519a4638686e42653d5
I20260807 18:20:10.908169 16440 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:20:10.918464 16440 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:20:10.919082 16440 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:20:10.939762 16440 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51c1fab3e4c24630a990ab8344349228: Generated new TSK 0
I20260807 18:20:10.940791 16440 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:20:10.956887 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 16331
W20260807 18:20:10.958482 16414 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.38:44145
W20260807 18:20:10.959573 17848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 16331: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:20:10.959746 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 16331
2026-08-07T18:20:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2371 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:20:11 dist-test-slave-39l8 krb5kdc[16461](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:20:11 dist-test-slave-39l8 krb5kdc[16461](info): set up 2 sockets
Aug 07 18:20:11 dist-test-slave-39l8 krb5kdc[16461](info): commencing operation
krb5kdc: starting...
W20260807 18:20:12.351631 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.337s user 0.003s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:20:12 dist-test-slave-39l8 krb5kdc[16461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:20:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:20:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:20:12.717995 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38661
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38661
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:20:13.091010 16482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:20:13.091346 16482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:20:13.091519 16482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:20:13.099291 16482 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:20:13.099433 16482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:20:13.099535 16482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:20:13.099632 16482 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:20:13.099745 16482 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:20:13.110742 16482 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38661
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:20:13.112341 16482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:20:13.113948 16482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:20:13.128073 16488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:13.128027 16490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:13.128096 16487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:20:13.128969 16482 server_base.cc:1034] running on GCE node
I20260807 18:20:13.130046 16482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:20:13.131973 16482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:20:13.133283 16482 hybrid_clock.cc:648] HybridClock initialized: now 1786126813133223 us; error 59 us; skew 500 ppm
I20260807 18:20:13.133723 16482 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:20:13 dist-test-slave-39l8 krb5kdc[16461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126813, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:20:13.137583 16482 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:20:13.139556 16482 webserver.cc:460] Webserver started at http://127.0.0.1:34765/ using document root <none> and password file <none>
I20260807 18:20:13.140374 16482 fs_manager.cc:359] Metadata directory not provided
I20260807 18:20:13.140502 16482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:20:13.140815 16482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:20:13.144568 16482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "96dc0f45dde74aff8f8c0edd1c35c33c"
format_stamp: "Formatted at 2026-08-07 18:20:13 on dist-test-slave-39l8"
server_key: "91706ea37aee640006aaaea72a7fcfbe"
server_key_iv: "2801b5aff5a0a592b7fee431a967b0e2"
server_key_version: "encryptionkey@0"
I20260807 18:20:13.145516 16482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "96dc0f45dde74aff8f8c0edd1c35c33c"
format_stamp: "Formatted at 2026-08-07 18:20:13 on dist-test-slave-39l8"
server_key: "91706ea37aee640006aaaea72a7fcfbe"
server_key_iv: "2801b5aff5a0a592b7fee431a967b0e2"
server_key_version: "encryptionkey@0"
I20260807 18:20:13.152072 16482 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 18:20:13.156780 16500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:13.158490 16482 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:20:13.158792 16482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "96dc0f45dde74aff8f8c0edd1c35c33c"
format_stamp: "Formatted at 2026-08-07 18:20:13 on dist-test-slave-39l8"
server_key: "91706ea37aee640006aaaea72a7fcfbe"
server_key_iv: "2801b5aff5a0a592b7fee431a967b0e2"
server_key_version: "encryptionkey@0"
I20260807 18:20:13.159019 16482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:20:13.171902 16482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:20:13.176872 16482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:20:13.177136 16482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:20:13.198027 16482 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38661
I20260807 18:20:13.198057 16579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38661 every 8 connection(s)
I20260807 18:20:13.199730 16482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:20:13.204384 16580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:20:13.205695 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 16482
I20260807 18:20:13.205891 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:20:13.206372 17848 external_mini_cluster.cc:1428] Setting key bb5a448950c44e2a2c80848d0055e594
I20260807 18:20:13.221666 16580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96dc0f45dde74aff8f8c0edd1c35c33c: Bootstrap starting.
Aug 07 18:20:13 dist-test-slave-39l8 krb5kdc[16461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 18:20:13.225406 16580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96dc0f45dde74aff8f8c0edd1c35c33c: Neither blocks nor log segments found. Creating new log.
I20260807 18:20:13.227097 16580 log.cc:824] T 00000000000000000000000000000000 P 96dc0f45dde74aff8f8c0edd1c35c33c: Log is configured to *not* fsync() on all Append() calls
I20260807 18:20:13.231242 16580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96dc0f45dde74aff8f8c0edd1c35c33c: No bootstrap required, opened a new log
I20260807 18:20:13.235396 16583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:13.208739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:37055 (local address 127.0.0.1:38661)
0807 18:20:13.209691 (+ 952us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:13.209711 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:13.209785 (+ 74us) server_negotiation.cc:407] Connection header received
0807 18:20:13.212591 (+ 2806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:13.212633 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:13.213316 (+ 683us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:13.213878 (+ 562us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:13.215547 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:13.217643 (+ 2096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:13.219251 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:13.220480 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:13.220704 (+ 224us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:13.225351 (+ 4647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:13.225366 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:13.225400 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:13.225459 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:13.229741 (+ 4282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:13.230823 (+ 1082us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:13.230833 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:13.230843 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:13.231080 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:13.231727 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:13.231734 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:13.231739 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:13.232547 (+ 808us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:13.232904 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:13.233370 (+ 466us) server_negotiation.cc:299] Negotiation successful
0807 18:20:13.233808 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":379,"threads_started":1}
I20260807 18:20:13.240232 17848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:20:13.241743 16580 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96dc0f45dde74aff8f8c0edd1c35c33c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96dc0f45dde74aff8f8c0edd1c35c33c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38661 } }
I20260807 18:20:13.242180 16580 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96dc0f45dde74aff8f8c0edd1c35c33c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:20:13.242290 16580 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96dc0f45dde74aff8f8c0edd1c35c33c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96dc0f45dde74aff8f8c0edd1c35c33c, State: Initialized, Role: FOLLOWER
I20260807 18:20:13.243168 16580 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96dc0f45dde74aff8f8c0edd1c35c33c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96dc0f45dde74aff8f8c0edd1c35c33c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38661 } }
I20260807 18:20:13.243468 16580 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96dc0f45dde74aff8f8c0edd1c35c33c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:20:13.243643 16580 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96dc0f45dde74aff8f8c0edd1c35c33c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:20:13.243861 16580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96dc0f45dde74aff8f8c0edd1c35c33c [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:20:13.246583 16580 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96dc0f45dde74aff8f8c0edd1c35c33c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96dc0f45dde74aff8f8c0edd1c35c33c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38661 } }
I20260807 18:20:13.247197 16580 leader_election.cc:302] T 00000000000000000000000000000000 P 96dc0f45dde74aff8f8c0edd1c35c33c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96dc0f45dde74aff8f8c0edd1c35c33c; no voters:
I20260807 18:20:13.247843 16580 leader_election.cc:288] T 00000000000000000000000000000000 P 96dc0f45dde74aff8f8c0edd1c35c33c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:20:13.248019 16595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96dc0f45dde74aff8f8c0edd1c35c33c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:20:13.250118 16580 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96dc0f45dde74aff8f8c0edd1c35c33c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:20:13.256435 16595 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96dc0f45dde74aff8f8c0edd1c35c33c [term 1 LEADER]: Becoming Leader. State: Replica: 96dc0f45dde74aff8f8c0edd1c35c33c, State: Running, Role: LEADER
I20260807 18:20:13.257097 16595 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96dc0f45dde74aff8f8c0edd1c35c33c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96dc0f45dde74aff8f8c0edd1c35c33c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38661 } }
I20260807 18:20:13.260315 16596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96dc0f45dde74aff8f8c0edd1c35c33c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96dc0f45dde74aff8f8c0edd1c35c33c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96dc0f45dde74aff8f8c0edd1c35c33c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38661 } } }
I20260807 18:20:13.260792 16596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96dc0f45dde74aff8f8c0edd1c35c33c [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:13.260380 16597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96dc0f45dde74aff8f8c0edd1c35c33c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96dc0f45dde74aff8f8c0edd1c35c33c. Latest consensus state: current_term: 1 leader_uuid: "96dc0f45dde74aff8f8c0edd1c35c33c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96dc0f45dde74aff8f8c0edd1c35c33c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38661 } } }
I20260807 18:20:13.261379 16597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96dc0f45dde74aff8f8c0edd1c35c33c [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:13.268594 16583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:13.245327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:36195 (local address 127.0.0.1:38661)
0807 18:20:13.245588 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:13.245601 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:13.246128 (+ 527us) server_negotiation.cc:407] Connection header received
0807 18:20:13.246354 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:13.246358 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:13.246424 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:13.246545 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:13.248700 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:13.250329 (+ 1629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:13.253434 (+ 3105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:13.254540 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:13.254752 (+ 212us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:13.258477 (+ 3725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:13.258483 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:13.258487 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:13.258528 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:13.263613 (+ 5085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:13.265803 (+ 2190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:13.265810 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:13.265813 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:13.265996 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:13.266690 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:13.266695 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:13.266698 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:13.267142 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:13.267451 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:13.267931 (+ 480us) server_negotiation.cc:299] Negotiation successful
0807 18:20:13.268150 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260807 18:20:13.271175 16617 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 96dc0f45dde74aff8f8c0edd1c35c33c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:20:13.271347 16617 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:20:13.271492 16614 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:20:13.273901 16614 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:20:13.285226 16614 catalog_manager.cc:1347] Generated new cluster ID: 80dc1287b38a420684a1396bf12e389f
I20260807 18:20:13.285411 16614 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:20:13.293398 16614 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:20:13.295387 16614 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:20:13.310331 16614 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96dc0f45dde74aff8f8c0edd1c35c33c: Generated new TSK 0
I20260807 18:20:13.311681 16614 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:20:13.320396 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 16482
I20260807 18:20:13.321827 16572 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:20:13.417009 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 16482
W20260807 18:20:13.445384 16457 connection.cc:535] client connection to 127.0.0.1:38661 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38661): Connection reset by peer (error 104)
2026-08-07T18:20:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2459 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:20:13 dist-test-slave-39l8 krb5kdc[16628](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:20:13 dist-test-slave-39l8 krb5kdc[16628](info): set up 2 sockets
Aug 07 18:20:13 dist-test-slave-39l8 krb5kdc[16628](info): commencing operation
krb5kdc: starting...
W20260807 18:20:14.793862 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.323s user 0.006s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:20:15 dist-test-slave-39l8 krb5kdc[16628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:20:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:20:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:20:15.180080 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41659
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41659
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:20:15.564143 16650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:20:15.564479 16650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:20:15.564642 16650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:20:15.572691 16650 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:20:15.572830 16650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:20:15.572912 16650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:20:15.573024 16650 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:20:15.573088 16650 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:20:15.584321 16650 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41659
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:20:15.585791 16650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:20:15.587437 16650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:20:15.600374 16659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:15.600414 16660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:15.600625 16662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:20:15.602072 16650 server_base.cc:1034] running on GCE node
I20260807 18:20:15.603251 16650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:20:15.605427 16650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:20:15.606781 16650 hybrid_clock.cc:648] HybridClock initialized: now 1786126815606614 us; error 134 us; skew 500 ppm
I20260807 18:20:15.607276 16650 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:20:15 dist-test-slave-39l8 krb5kdc[16628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126815, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:20:15.611228 16650 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:20:15.613118 16650 webserver.cc:460] Webserver started at http://127.0.0.1:46643/ using document root <none> and password file <none>
I20260807 18:20:15.613935 16650 fs_manager.cc:359] Metadata directory not provided
I20260807 18:20:15.614118 16650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:20:15.614463 16650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:20:15.617815 16650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "590c74ef9d604a13875f447e77186558"
format_stamp: "Formatted at 2026-08-07 18:20:15 on dist-test-slave-39l8"
server_key: "f4b4aae5034d95a7484ce8abc74c3cca"
server_key_iv: "10d8397489b35e19120e99306c0446f7"
server_key_version: "encryptionkey@0"
I20260807 18:20:15.618774 16650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "590c74ef9d604a13875f447e77186558"
format_stamp: "Formatted at 2026-08-07 18:20:15 on dist-test-slave-39l8"
server_key: "f4b4aae5034d95a7484ce8abc74c3cca"
server_key_iv: "10d8397489b35e19120e99306c0446f7"
server_key_version: "encryptionkey@0"
I20260807 18:20:15.624825 16650 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 18:20:15.628921 16674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:15.630515 16650 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:20:15.630817 16650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "590c74ef9d604a13875f447e77186558"
format_stamp: "Formatted at 2026-08-07 18:20:15 on dist-test-slave-39l8"
server_key: "f4b4aae5034d95a7484ce8abc74c3cca"
server_key_iv: "10d8397489b35e19120e99306c0446f7"
server_key_version: "encryptionkey@0"
I20260807 18:20:15.631060 16650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:20:15.669111 16650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:20:15.674294 16650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:20:15.674641 16650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:20:15.695454 16650 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41659
I20260807 18:20:15.695497 16742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41659 every 8 connection(s)
I20260807 18:20:15.696965 16650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:20:15.701071 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 16650
I20260807 18:20:15.701292 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:20:15.701715 16745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:20:15.701897 17848 external_mini_cluster.cc:1428] Setting key de9e80cf2967bf8d6266c281ed6616e0
I20260807 18:20:15.717381 16749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:15.703898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:59255 (local address 127.0.0.1:41659)
0807 18:20:15.704812 (+ 914us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:15.704828 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:15.704873 (+ 45us) server_negotiation.cc:407] Connection header received
0807 18:20:15.706953 (+ 2080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:15.706982 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:15.707520 (+ 538us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:15.707909 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:15.708665 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:15.710138 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:15.712158 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:15.712942 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:15.713091 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:15.714606 (+ 1515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:15.714614 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:15.714632 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:20:15.715378 (+ 746us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:15.715473 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:20:15.715512 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:15.715648 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:15.716050 (+ 402us) server_negotiation.cc:299] Negotiation successful
0807 18:20:15.716314 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":392,"threads_started":1}
I20260807 18:20:15.717661 16745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 590c74ef9d604a13875f447e77186558: Bootstrap starting.
I20260807 18:20:15.721854 16745 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 590c74ef9d604a13875f447e77186558: Neither blocks nor log segments found. Creating new log.
I20260807 18:20:15.722119 17848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:20:15.723124 16745 log.cc:824] T 00000000000000000000000000000000 P 590c74ef9d604a13875f447e77186558: Log is configured to *not* fsync() on all Append() calls
I20260807 18:20:15.726698 16745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 590c74ef9d604a13875f447e77186558: No bootstrap required, opened a new log
I20260807 18:20:15.734216 16749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:15.727158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:48237 (local address 127.0.0.1:41659)
0807 18:20:15.727382 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:15.727386 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:15.727928 (+ 542us) server_negotiation.cc:407] Connection header received
0807 18:20:15.728177 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:15.728182 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:15.728261 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:15.728397 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:15.728974 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:15.729826 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:15.731392 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:15.731904 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:15.732089 (+ 185us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:15.733369 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:15.733374 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:15.733377 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:20:15.733398 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:15.733467 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:20:15.733488 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:15.733639 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:15.733840 (+ 201us) server_negotiation.cc:299] Negotiation successful
0807 18:20:15.733903 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260807 18:20:15.735697 16745 raft_consensus.cc:348] T 00000000000000000000000000000000 P 590c74ef9d604a13875f447e77186558 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "590c74ef9d604a13875f447e77186558" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41659 } }
I20260807 18:20:15.736176 16745 raft_consensus.cc:374] T 00000000000000000000000000000000 P 590c74ef9d604a13875f447e77186558 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:20:15.736323 16745 raft_consensus.cc:729] T 00000000000000000000000000000000 P 590c74ef9d604a13875f447e77186558 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 590c74ef9d604a13875f447e77186558, State: Initialized, Role: FOLLOWER
I20260807 18:20:15.736938 16745 consensus_queue.cc:260] T 00000000000000000000000000000000 P 590c74ef9d604a13875f447e77186558 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "590c74ef9d604a13875f447e77186558" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41659 } }
I20260807 18:20:15.737205 16745 raft_consensus.cc:388] T 00000000000000000000000000000000 P 590c74ef9d604a13875f447e77186558 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:20:15.737375 16745 raft_consensus.cc:482] T 00000000000000000000000000000000 P 590c74ef9d604a13875f447e77186558 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:20:15.737584 16745 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 590c74ef9d604a13875f447e77186558 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:20:15.740119 16745 raft_consensus.cc:504] T 00000000000000000000000000000000 P 590c74ef9d604a13875f447e77186558 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "590c74ef9d604a13875f447e77186558" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41659 } }
I20260807 18:20:15.740675 16745 leader_election.cc:302] T 00000000000000000000000000000000 P 590c74ef9d604a13875f447e77186558 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 590c74ef9d604a13875f447e77186558; no voters:
I20260807 18:20:15.741314 16745 leader_election.cc:288] T 00000000000000000000000000000000 P 590c74ef9d604a13875f447e77186558 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:20:15.741503 16762 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 590c74ef9d604a13875f447e77186558 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:20:15.742302 16762 raft_consensus.cc:686] T 00000000000000000000000000000000 P 590c74ef9d604a13875f447e77186558 [term 1 LEADER]: Becoming Leader. State: Replica: 590c74ef9d604a13875f447e77186558, State: Running, Role: LEADER
I20260807 18:20:15.742959 16762 consensus_queue.cc:237] T 00000000000000000000000000000000 P 590c74ef9d604a13875f447e77186558 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "590c74ef9d604a13875f447e77186558" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41659 } }
I20260807 18:20:15.743757 16745 sys_catalog.cc:564] T 00000000000000000000000000000000 P 590c74ef9d604a13875f447e77186558 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:20:15.746073 16764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 590c74ef9d604a13875f447e77186558 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "590c74ef9d604a13875f447e77186558" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "590c74ef9d604a13875f447e77186558" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41659 } } }
I20260807 18:20:15.746587 16764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 590c74ef9d604a13875f447e77186558 [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:15.746613 16765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 590c74ef9d604a13875f447e77186558 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 590c74ef9d604a13875f447e77186558. Latest consensus state: current_term: 1 leader_uuid: "590c74ef9d604a13875f447e77186558" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "590c74ef9d604a13875f447e77186558" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41659 } } }
I20260807 18:20:15.746954 16765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 590c74ef9d604a13875f447e77186558 [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:15.749246 16772 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:20:15.755548 16772 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:20:15.765249 16772 catalog_manager.cc:1347] Generated new cluster ID: ffebf0d5865246f3900f43caef67062d
I20260807 18:20:15.765393 16772 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:20:15.787802 16772 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:20:15.789249 16772 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:20:15.796098 16772 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 590c74ef9d604a13875f447e77186558: Generated new TSK 0
I20260807 18:20:15.797027 16772 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:20:15.809674 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 16650
W20260807 18:20:15.812044 16737 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.38:59255
W20260807 18:20:15.812979 17848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 16650: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:20:15.813180 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 16650
W20260807 18:20:15.841257 16626 connection.cc:535] client connection to 127.0.0.1:41659 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41659): Connection reset by peer (error 104)
2026-08-07T18:20:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2390 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:20:15 dist-test-slave-39l8 krb5kdc[16791](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:20:15 dist-test-slave-39l8 krb5kdc[16791](info): set up 2 sockets
Aug 07 18:20:15 dist-test-slave-39l8 krb5kdc[16791](info): commencing operation
krb5kdc: starting...
W20260807 18:20:17.216624 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.355s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:20:17 dist-test-slave-39l8 krb5kdc[16791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:20:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:20:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:20:17.583344 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38889
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:20:17.962356 16810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:20:17.962678 16810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:20:17.962877 16810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:20:17.970649 16810 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:20:17.970861 16810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:20:17.970969 16810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:20:17.971055 16810 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:20:17.971122 16810 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:20:17.981775 16810 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38889
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126429568977-17848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:20:17.983433 16810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:20:17.984860 16810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:20:17.998354 16821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:17.998548 16820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:17.998512 16823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:20:17.999329 16810 server_base.cc:1034] running on GCE node
I20260807 18:20:18.000272 16810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:20:18.002022 16810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:20:18.003304 16810 hybrid_clock.cc:648] HybridClock initialized: now 1786126818003247 us; error 71 us; skew 500 ppm
I20260807 18:20:18.003754 16810 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:20:18 dist-test-slave-39l8 krb5kdc[16791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126818, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:20:18.007757 16810 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:20:18.009532 16810 webserver.cc:460] Webserver started at http://127.0.0.1:43639/ using document root <none> and password file <none>
I20260807 18:20:18.010339 16810 fs_manager.cc:359] Metadata directory not provided
I20260807 18:20:18.010506 16810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:20:18.010941 16810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:20:18.014398 16810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "029e41ff75d04469bf98d5cdaafe4752"
format_stamp: "Formatted at 2026-08-07 18:20:18 on dist-test-slave-39l8"
server_key: "d32310c66e8b2f971cdf8db735754860"
server_key_iv: "35615440194ee961dd0fe4ccf3d04bb6"
server_key_version: "encryptionkey@0"
I20260807 18:20:18.015282 16810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "029e41ff75d04469bf98d5cdaafe4752"
format_stamp: "Formatted at 2026-08-07 18:20:18 on dist-test-slave-39l8"
server_key: "d32310c66e8b2f971cdf8db735754860"
server_key_iv: "35615440194ee961dd0fe4ccf3d04bb6"
server_key_version: "encryptionkey@0"
I20260807 18:20:18.024714 16810 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.006s sys 0.001s
I20260807 18:20:18.029199 16832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:18.031035 16810 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:20:18.031304 16810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "029e41ff75d04469bf98d5cdaafe4752"
format_stamp: "Formatted at 2026-08-07 18:20:18 on dist-test-slave-39l8"
server_key: "d32310c66e8b2f971cdf8db735754860"
server_key_iv: "35615440194ee961dd0fe4ccf3d04bb6"
server_key_version: "encryptionkey@0"
I20260807 18:20:18.031571 16810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:20:18.039007 16810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:20:18.044101 16810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:20:18.044497 16810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:20:18.067641 16810 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38889
I20260807 18:20:18.067683 16899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38889 every 8 connection(s)
I20260807 18:20:18.069437 16810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:20:18.071333 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 16810
I20260807 18:20:18.071547 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:20:18.072204 17848 external_mini_cluster.cc:1428] Setting key f9093aec44a105bd36f5a79d1f5f624a
I20260807 18:20:18.075107 16900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:20:18.081539 16903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:18.074369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:53709 (local address 127.0.0.1:38889)
0807 18:20:18.075451 (+ 1082us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:18.075469 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:18.075518 (+ 49us) server_negotiation.cc:407] Connection header received
0807 18:20:18.077327 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:18.077351 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:18.077746 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:18.078176 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:18.079553 (+ 1377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:18.079561 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:18.079577 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:20:18.079596 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:18.079703 (+ 107us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:20:18.079729 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:18.079936 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:18.080057 (+ 121us) server_negotiation.cc:299] Negotiation successful
0807 18:20:18.080351 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":810,"thread_start_us":521,"threads_started":1}
I20260807 18:20:18.086226 17848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:20:18.089638 16900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 029e41ff75d04469bf98d5cdaafe4752: Bootstrap starting.
I20260807 18:20:18.094528 16900 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 029e41ff75d04469bf98d5cdaafe4752: Neither blocks nor log segments found. Creating new log.
I20260807 18:20:18.094635 16903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:18.091761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:47531 (local address 127.0.0.1:38889)
0807 18:20:18.091999 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:18.092022 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:18.092358 (+ 336us) server_negotiation.cc:407] Connection header received
0807 18:20:18.092571 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:18.092574 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:18.092633 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:18.092737 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:18.093830 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:18.093834 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:18.093838 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:20:18.093855 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:18.093898 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:20:18.093916 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:18.094018 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:18.094188 (+ 170us) server_negotiation.cc:299] Negotiation successful
0807 18:20:18.094246 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260807 18:20:18.095930 16900 log.cc:824] T 00000000000000000000000000000000 P 029e41ff75d04469bf98d5cdaafe4752: Log is configured to *not* fsync() on all Append() calls
I20260807 18:20:18.099331 16900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 029e41ff75d04469bf98d5cdaafe4752: No bootstrap required, opened a new log
I20260807 18:20:18.108489 16900 raft_consensus.cc:348] T 00000000000000000000000000000000 P 029e41ff75d04469bf98d5cdaafe4752 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "029e41ff75d04469bf98d5cdaafe4752" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38889 } }
I20260807 18:20:18.109027 16900 raft_consensus.cc:374] T 00000000000000000000000000000000 P 029e41ff75d04469bf98d5cdaafe4752 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:20:18.109158 16900 raft_consensus.cc:729] T 00000000000000000000000000000000 P 029e41ff75d04469bf98d5cdaafe4752 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 029e41ff75d04469bf98d5cdaafe4752, State: Initialized, Role: FOLLOWER
I20260807 18:20:18.109866 16900 consensus_queue.cc:260] T 00000000000000000000000000000000 P 029e41ff75d04469bf98d5cdaafe4752 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "029e41ff75d04469bf98d5cdaafe4752" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38889 } }
I20260807 18:20:18.110163 16900 raft_consensus.cc:388] T 00000000000000000000000000000000 P 029e41ff75d04469bf98d5cdaafe4752 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:20:18.110347 16900 raft_consensus.cc:482] T 00000000000000000000000000000000 P 029e41ff75d04469bf98d5cdaafe4752 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:20:18.110553 16900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 029e41ff75d04469bf98d5cdaafe4752 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:20:18.113256 16900 raft_consensus.cc:504] T 00000000000000000000000000000000 P 029e41ff75d04469bf98d5cdaafe4752 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "029e41ff75d04469bf98d5cdaafe4752" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38889 } }
I20260807 18:20:18.114111 16900 leader_election.cc:302] T 00000000000000000000000000000000 P 029e41ff75d04469bf98d5cdaafe4752 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 029e41ff75d04469bf98d5cdaafe4752; no voters:
I20260807 18:20:18.114887 16900 leader_election.cc:288] T 00000000000000000000000000000000 P 029e41ff75d04469bf98d5cdaafe4752 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:20:18.115116 16914 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 029e41ff75d04469bf98d5cdaafe4752 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:20:18.115954 16914 raft_consensus.cc:686] T 00000000000000000000000000000000 P 029e41ff75d04469bf98d5cdaafe4752 [term 1 LEADER]: Becoming Leader. State: Replica: 029e41ff75d04469bf98d5cdaafe4752, State: Running, Role: LEADER
I20260807 18:20:18.116668 16914 consensus_queue.cc:237] T 00000000000000000000000000000000 P 029e41ff75d04469bf98d5cdaafe4752 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "029e41ff75d04469bf98d5cdaafe4752" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38889 } }
I20260807 18:20:18.117416 16900 sys_catalog.cc:564] T 00000000000000000000000000000000 P 029e41ff75d04469bf98d5cdaafe4752 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:20:18.120406 16915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 029e41ff75d04469bf98d5cdaafe4752 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "029e41ff75d04469bf98d5cdaafe4752" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "029e41ff75d04469bf98d5cdaafe4752" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38889 } } }
I20260807 18:20:18.121106 16915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 029e41ff75d04469bf98d5cdaafe4752 [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:18.120484 16916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 029e41ff75d04469bf98d5cdaafe4752 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 029e41ff75d04469bf98d5cdaafe4752. Latest consensus state: current_term: 1 leader_uuid: "029e41ff75d04469bf98d5cdaafe4752" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "029e41ff75d04469bf98d5cdaafe4752" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38889 } } }
I20260807 18:20:18.121794 16916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 029e41ff75d04469bf98d5cdaafe4752 [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:18.124070 16924 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:20:18.130614 16924 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:20:18.141114 16924 catalog_manager.cc:1347] Generated new cluster ID: 613b6bb68c4a47e89ce3574e9aaffc53
I20260807 18:20:18.141263 16924 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:20:18.170850 16924 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:20:18.171437 16924 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:20:18.196249 16924 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 029e41ff75d04469bf98d5cdaafe4752: Generated new TSK 0
I20260807 18:20:18.197423 16924 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:20:18.205835 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 16810
W20260807 18:20:18.207325 16895 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.38:53709
W20260807 18:20:18.208230 17848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 16810: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:20:18.208432 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 16810
2026-08-07T18:20:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2397 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:20:18 dist-test-slave-39l8 krb5kdc[16942](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:20:18 dist-test-slave-39l8 krb5kdc[16942](info): set up 2 sockets
Aug 07 18:20:18 dist-test-slave-39l8 krb5kdc[16942](info): commencing operation
krb5kdc: starting...
W20260807 18:20:19.554904 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.295s user 0.002s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:20:19 dist-test-slave-39l8 krb5kdc[16942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:20:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:20:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.110.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
I20260807 18:20:19.921792 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:37451
--webserver_interface=127.17.110.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:38559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:37451
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:20:20.300544 16962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:20:20.300885 16962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:20:20.301095 16962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:20:20.308948 16962 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:20:20.309098 16962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:20:20.309217 16962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:20:20.309342 16962 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:20:20.309417 16962 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:20:20.320264 16962 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:38559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:37451
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:37451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:20:20.321795 16962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:20:20.323334 16962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:20:20.337425 16973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:20.337464 16969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:20.337489 16968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:20:20.338171 16962 server_base.cc:1034] running on GCE node
I20260807 18:20:20.339345 16962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:20:20.341123 16962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:20:20.342447 16962 hybrid_clock.cc:648] HybridClock initialized: now 1786126820342392 us; error 51 us; skew 500 ppm
I20260807 18:20:20.342964 16962 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:20:20 dist-test-slave-39l8 krb5kdc[16942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126820, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:20:20.347105 16962 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:20:20.349272 16962 webserver.cc:460] Webserver started at http://127.17.110.62:35551/ using document root <none> and password file <none>
I20260807 18:20:20.350222 16962 fs_manager.cc:359] Metadata directory not provided
I20260807 18:20:20.350415 16962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:20:20.350881 16962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:20:20.354674 16962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "ef15a897c2744120976b0a7b2150fcd0"
format_stamp: "Formatted at 2026-08-07 18:20:20 on dist-test-slave-39l8"
server_key: "a0ca425539cd9343b820c58dfb8b65d7"
server_key_iv: "7a6bb6636e3a52a882910197034cb36d"
server_key_version: "encryptionkey@0"
I20260807 18:20:20.355804 16962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "ef15a897c2744120976b0a7b2150fcd0"
format_stamp: "Formatted at 2026-08-07 18:20:20 on dist-test-slave-39l8"
server_key: "a0ca425539cd9343b820c58dfb8b65d7"
server_key_iv: "7a6bb6636e3a52a882910197034cb36d"
server_key_version: "encryptionkey@0"
I20260807 18:20:20.362425 16962 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 18:20:20.367290 16985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:20.369076 16962 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:20:20.369372 16962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "ef15a897c2744120976b0a7b2150fcd0"
format_stamp: "Formatted at 2026-08-07 18:20:20 on dist-test-slave-39l8"
server_key: "a0ca425539cd9343b820c58dfb8b65d7"
server_key_iv: "7a6bb6636e3a52a882910197034cb36d"
server_key_version: "encryptionkey@0"
I20260807 18:20:20.369642 16962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:20:20.388967 16962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:20:20.394227 16962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:20:20.394609 16962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:20:20.416347 16962 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:37451
I20260807 18:20:20.416385 17053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:37451 every 8 connection(s)
I20260807 18:20:20.417893 16962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:20:20.419446 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 16962
I20260807 18:20:20.419698 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:20:20.420295 17848 external_mini_cluster.cc:1428] Setting key 8ae0687f13e7b969920aefa7d1a14ffd
I20260807 18:20:20.423668 17056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 18:20:20 dist-test-slave-39l8 krb5kdc[16942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:20:20.437451 17056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef15a897c2744120976b0a7b2150fcd0: Bootstrap starting.
I20260807 18:20:20.442016 17056 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef15a897c2744120976b0a7b2150fcd0: Neither blocks nor log segments found. Creating new log.
I20260807 18:20:20.443424 17056 log.cc:824] T 00000000000000000000000000000000 P ef15a897c2744120976b0a7b2150fcd0: Log is configured to *not* fsync() on all Append() calls
I20260807 18:20:20.446929 17056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef15a897c2744120976b0a7b2150fcd0: No bootstrap required, opened a new log
I20260807 18:20:20.447824 17060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:20.422404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34259 (local address 127.17.110.62:37451)
0807 18:20:20.423328 (+ 924us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:20.423342 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:20.423388 (+ 46us) server_negotiation.cc:407] Connection header received
0807 18:20:20.425262 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:20.425290 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:20.425744 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:20.426196 (+ 452us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:20.427982 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:20.429512 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:20.431144 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:20.432002 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:20.432256 (+ 254us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:20.437837 (+ 5581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:20.437849 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:20.437868 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:20.437930 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:20.442246 (+ 4316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:20.443493 (+ 1247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:20.443502 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:20.443512 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:20.443801 (+ 289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:20.444443 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:20.444451 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:20.444453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:20.445102 (+ 649us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:20.445482 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:20.446096 (+ 614us) server_negotiation.cc:299] Negotiation successful
0807 18:20:20.446473 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":437,"threads_started":1}
I20260807 18:20:20.453202 17848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:20:20.456468 17056 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef15a897c2744120976b0a7b2150fcd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef15a897c2744120976b0a7b2150fcd0" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 37451 } }
I20260807 18:20:20.457118 17056 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef15a897c2744120976b0a7b2150fcd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:20:20.457320 17056 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef15a897c2744120976b0a7b2150fcd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef15a897c2744120976b0a7b2150fcd0, State: Initialized, Role: FOLLOWER
I20260807 18:20:20.458231 17056 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef15a897c2744120976b0a7b2150fcd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef15a897c2744120976b0a7b2150fcd0" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 37451 } }
I20260807 18:20:20.458523 17056 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef15a897c2744120976b0a7b2150fcd0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:20:20.458781 17056 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef15a897c2744120976b0a7b2150fcd0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:20:20.459061 17056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef15a897c2744120976b0a7b2150fcd0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:20:20.462877 17056 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef15a897c2744120976b0a7b2150fcd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef15a897c2744120976b0a7b2150fcd0" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 37451 } }
I20260807 18:20:20.463551 17056 leader_election.cc:302] T 00000000000000000000000000000000 P ef15a897c2744120976b0a7b2150fcd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef15a897c2744120976b0a7b2150fcd0; no voters:
I20260807 18:20:20.464385 17056 leader_election.cc:288] T 00000000000000000000000000000000 P ef15a897c2744120976b0a7b2150fcd0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:20:20.464558 17070 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef15a897c2744120976b0a7b2150fcd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:20:20.466048 17070 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef15a897c2744120976b0a7b2150fcd0 [term 1 LEADER]: Becoming Leader. State: Replica: ef15a897c2744120976b0a7b2150fcd0, State: Running, Role: LEADER
I20260807 18:20:20.466858 17070 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef15a897c2744120976b0a7b2150fcd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef15a897c2744120976b0a7b2150fcd0" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 37451 } }
I20260807 18:20:20.467528 17056 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef15a897c2744120976b0a7b2150fcd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:20:20.471316 17072 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef15a897c2744120976b0a7b2150fcd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef15a897c2744120976b0a7b2150fcd0. Latest consensus state: current_term: 1 leader_uuid: "ef15a897c2744120976b0a7b2150fcd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef15a897c2744120976b0a7b2150fcd0" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 37451 } } }
I20260807 18:20:20.471836 17072 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef15a897c2744120976b0a7b2150fcd0 [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:20.474596 17071 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef15a897c2744120976b0a7b2150fcd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef15a897c2744120976b0a7b2150fcd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef15a897c2744120976b0a7b2150fcd0" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 37451 } } }
I20260807 18:20:20.475404 17071 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef15a897c2744120976b0a7b2150fcd0 [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:20.477599 17081 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:20:20.482455 17060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:20.458979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42191 (local address 127.17.110.62:37451)
0807 18:20:20.459272 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:20.459284 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:20.459854 (+ 570us) server_negotiation.cc:407] Connection header received
0807 18:20:20.460047 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:20.460052 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:20.460130 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:20.460314 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:20.462452 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:20.463551 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:20.465841 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:20.466675 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:20.466879 (+ 204us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:20.471105 (+ 4226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:20.471117 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:20.471125 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:20.471189 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:20.476277 (+ 5088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:20.477730 (+ 1453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:20.477740 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:20.477751 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:20.478102 (+ 351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:20.479002 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:20.479008 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:20.479012 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:20.479493 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:20.481285 (+ 1792us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:20.481563 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 18:20:20.481881 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:20:20.486122 17081 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:20:20.497303 17081 catalog_manager.cc:1347] Generated new cluster ID: b2d2ada8713241a88a607baff94bd4cd
I20260807 18:20:20.497452 17081 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:20:20.507249 17081 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:20:20.509426 17081 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:20:20.523308 17081 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef15a897c2744120976b0a7b2150fcd0: Generated new TSK 0
I20260807 18:20:20.524425 17081 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:20:20.530476 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 16962
I20260807 18:20:20.532332 17047 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:20:20.635824 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 16962
W20260807 18:20:20.664608 16938 connection.cc:535] client connection to 127.17.110.62:37451 recv error: Network error: failed to read from TLS socket (remote: 127.17.110.62:37451): Connection reset by peer (error 104)
2026-08-07T18:20:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2425 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:20:20 dist-test-slave-39l8 krb5kdc[17099](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:20:20 dist-test-slave-39l8 krb5kdc[17099](info): set up 2 sockets
Aug 07 18:20:20 dist-test-slave-39l8 krb5kdc[17099](info): commencing operation
krb5kdc: starting...
W20260807 18:20:22.037618 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.352s user 0.002s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:20:22 dist-test-slave-39l8 krb5kdc[17099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:20:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:20:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.110.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
I20260807 18:20:22.408324 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:37801
--webserver_interface=127.17.110.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:45267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:37801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:20:22.803846 17114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:20:22.804225 17114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:20:22.804463 17114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:20:22.813146 17114 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:20:22.813308 17114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:20:22.813439 17114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:20:22.813570 17114 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:20:22.813663 17114 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:20:22.825011 17114 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:45267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:37801
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:37801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:20:22.826509 17114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:20:22.827981 17114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:20:22.842202 17120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:22.842320 17121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:22.842327 17124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:20:22.843572 17114 server_base.cc:1034] running on GCE node
I20260807 18:20:22.844607 17114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:20:22.846278 17114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:20:22.847610 17114 hybrid_clock.cc:648] HybridClock initialized: now 1786126822847521 us; error 51 us; skew 500 ppm
I20260807 18:20:22.848040 17114 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:20:22 dist-test-slave-39l8 krb5kdc[17099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126822, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:20:22.851768 17114 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:20:22.853610 17114 webserver.cc:460] Webserver started at http://127.17.110.62:39737/ using document root <none> and password file <none>
I20260807 18:20:22.854565 17114 fs_manager.cc:359] Metadata directory not provided
I20260807 18:20:22.854797 17114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:20:22.855240 17114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:20:22.859048 17114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "448af13fcd744e429855421d90246598"
format_stamp: "Formatted at 2026-08-07 18:20:22 on dist-test-slave-39l8"
server_key: "95f9d5db29f1121a47cbdc4ed66a5aac"
server_key_iv: "e3cdbb1376c02503a6691139759a1533"
server_key_version: "encryptionkey@0"
I20260807 18:20:22.860062 17114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "448af13fcd744e429855421d90246598"
format_stamp: "Formatted at 2026-08-07 18:20:22 on dist-test-slave-39l8"
server_key: "95f9d5db29f1121a47cbdc4ed66a5aac"
server_key_iv: "e3cdbb1376c02503a6691139759a1533"
server_key_version: "encryptionkey@0"
I20260807 18:20:22.866636 17114 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 18:20:22.871407 17134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:22.873350 17114 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:20:22.873644 17114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "448af13fcd744e429855421d90246598"
format_stamp: "Formatted at 2026-08-07 18:20:22 on dist-test-slave-39l8"
server_key: "95f9d5db29f1121a47cbdc4ed66a5aac"
server_key_iv: "e3cdbb1376c02503a6691139759a1533"
server_key_version: "encryptionkey@0"
I20260807 18:20:22.873930 17114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:20:22.911891 17114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:20:22.917965 17114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:20:22.918356 17114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:20:22.939968 17114 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:37801
I20260807 18:20:22.940081 17203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:37801 every 8 connection(s)
I20260807 18:20:22.941506 17114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:20:22.946272 17204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:20:22.946764 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 17114
I20260807 18:20:22.946998 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:20:22.947503 17848 external_mini_cluster.cc:1428] Setting key bfd3fff103db38306de1f664fc407086
I20260807 18:20:22.961664 17204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 448af13fcd744e429855421d90246598: Bootstrap starting.
I20260807 18:20:22.964036 17207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:22.949622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46291 (local address 127.17.110.62:37801)
0807 18:20:22.950679 (+ 1057us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:22.950694 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:22.950808 (+ 114us) server_negotiation.cc:407] Connection header received
0807 18:20:22.953204 (+ 2396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:22.953242 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:22.953779 (+ 537us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:22.954192 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:22.954915 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:22.956267 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:22.958390 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:22.959503 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:22.959747 (+ 244us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:22.961370 (+ 1623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:22.961380 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:22.961399 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:20:22.962071 (+ 672us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:22.962169 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:20:22.962202 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:22.962389 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:22.962799 (+ 410us) server_negotiation.cc:299] Negotiation successful
0807 18:20:22.962974 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":801,"thread_start_us":503,"threads_started":1}
I20260807 18:20:22.965423 17204 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 448af13fcd744e429855421d90246598: Neither blocks nor log segments found. Creating new log.
I20260807 18:20:22.966763 17204 log.cc:824] T 00000000000000000000000000000000 P 448af13fcd744e429855421d90246598: Log is configured to *not* fsync() on all Append() calls
I20260807 18:20:22.968784 17848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:20:22.970410 17204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 448af13fcd744e429855421d90246598: No bootstrap required, opened a new log
I20260807 18:20:22.979910 17204 raft_consensus.cc:348] T 00000000000000000000000000000000 P 448af13fcd744e429855421d90246598 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "448af13fcd744e429855421d90246598" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 37801 } }
I20260807 18:20:22.980425 17204 raft_consensus.cc:374] T 00000000000000000000000000000000 P 448af13fcd744e429855421d90246598 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:20:22.980567 17204 raft_consensus.cc:729] T 00000000000000000000000000000000 P 448af13fcd744e429855421d90246598 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 448af13fcd744e429855421d90246598, State: Initialized, Role: FOLLOWER
I20260807 18:20:22.981312 17204 consensus_queue.cc:260] T 00000000000000000000000000000000 P 448af13fcd744e429855421d90246598 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "448af13fcd744e429855421d90246598" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 37801 } }
I20260807 18:20:22.981603 17204 raft_consensus.cc:388] T 00000000000000000000000000000000 P 448af13fcd744e429855421d90246598 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:20:22.981802 17204 raft_consensus.cc:482] T 00000000000000000000000000000000 P 448af13fcd744e429855421d90246598 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:20:22.982010 17204 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 448af13fcd744e429855421d90246598 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:20:22.982291 17207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:22.974259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41931 (local address 127.17.110.62:37801)
0807 18:20:22.974546 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:22.974557 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:22.974995 (+ 438us) server_negotiation.cc:407] Connection header received
0807 18:20:22.975293 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:22.975298 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:22.975374 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:22.975521 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:22.976216 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:22.977149 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:22.978887 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:22.979414 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:22.979603 (+ 189us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:22.981205 (+ 1602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:22.981211 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:22.981215 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:20:22.981238 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:22.981312 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:20:22.981332 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:22.981490 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:22.981750 (+ 260us) server_negotiation.cc:299] Negotiation successful
0807 18:20:22.981833 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 18:20:22.985040 17204 raft_consensus.cc:504] T 00000000000000000000000000000000 P 448af13fcd744e429855421d90246598 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "448af13fcd744e429855421d90246598" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 37801 } }
I20260807 18:20:22.985621 17204 leader_election.cc:302] T 00000000000000000000000000000000 P 448af13fcd744e429855421d90246598 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 448af13fcd744e429855421d90246598; no voters:
I20260807 18:20:22.986204 17204 leader_election.cc:288] T 00000000000000000000000000000000 P 448af13fcd744e429855421d90246598 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:20:22.986400 17215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 448af13fcd744e429855421d90246598 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:20:22.987783 17215 raft_consensus.cc:686] T 00000000000000000000000000000000 P 448af13fcd744e429855421d90246598 [term 1 LEADER]: Becoming Leader. State: Replica: 448af13fcd744e429855421d90246598, State: Running, Role: LEADER
I20260807 18:20:22.988443 17215 consensus_queue.cc:237] T 00000000000000000000000000000000 P 448af13fcd744e429855421d90246598 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "448af13fcd744e429855421d90246598" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 37801 } }
I20260807 18:20:22.988628 17204 sys_catalog.cc:564] T 00000000000000000000000000000000 P 448af13fcd744e429855421d90246598 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:20:22.991688 17218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 448af13fcd744e429855421d90246598 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 448af13fcd744e429855421d90246598. Latest consensus state: current_term: 1 leader_uuid: "448af13fcd744e429855421d90246598" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "448af13fcd744e429855421d90246598" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 37801 } } }
I20260807 18:20:22.992249 17218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 448af13fcd744e429855421d90246598 [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:22.992902 17216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 448af13fcd744e429855421d90246598 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "448af13fcd744e429855421d90246598" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "448af13fcd744e429855421d90246598" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 37801 } } }
I20260807 18:20:22.993326 17216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 448af13fcd744e429855421d90246598 [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:22.993901 17224 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:20:23.000700 17224 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:20:23.012087 17224 catalog_manager.cc:1347] Generated new cluster ID: ef4104d228254eacb2c4be0903a7725d
I20260807 18:20:23.012266 17224 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:20:23.022789 17224 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:20:23.024308 17224 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:20:23.037087 17224 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 448af13fcd744e429855421d90246598: Generated new TSK 0
I20260807 18:20:23.038041 17224 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:20:23.057603 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 17114
W20260807 18:20:23.059805 17197 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46291
W20260807 18:20:23.060914 17848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 17114: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:20:23.061192 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 17114
W20260807 18:20:23.087604 17095 connection.cc:535] client connection to 127.17.110.62:37801 recv error: Network error: failed to read from TLS socket (remote: 127.17.110.62:37801): Connection reset by peer (error 104)
2026-08-07T18:20:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2422 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:20:23 dist-test-slave-39l8 krb5kdc[17242](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:20:23 dist-test-slave-39l8 krb5kdc[17242](info): set up 2 sockets
Aug 07 18:20:23 dist-test-slave-39l8 krb5kdc[17242](info): commencing operation
krb5kdc: starting...
W20260807 18:20:24.491122 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.383s user 0.000s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:20:24 dist-test-slave-39l8 krb5kdc[17242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126824, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:20:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:20:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.110.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
I20260807 18:20:24.855119 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:43205
--webserver_interface=127.17.110.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:41777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:43205
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:20:25.233202 17259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:20:25.233561 17259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:20:25.233747 17259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:20:25.241596 17259 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:20:25.241748 17259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:20:25.241847 17259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:20:25.241968 17259 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:20:25.242034 17259 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:20:25.252976 17259 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:41777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:43205
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:43205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:20:25.254499 17259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:20:25.256036 17259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:20:25.269802 17264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:25.269843 17269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:25.269796 17265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:20:25.270726 17259 server_base.cc:1034] running on GCE node
I20260807 18:20:25.271685 17259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:20:25.273360 17259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:20:25.274663 17259 hybrid_clock.cc:648] HybridClock initialized: now 1786126825274603 us; error 58 us; skew 500 ppm
I20260807 18:20:25.275182 17259 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:20:25 dist-test-slave-39l8 krb5kdc[17242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126825, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:20:25.278944 17259 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:20:25.281078 17259 webserver.cc:460] Webserver started at http://127.17.110.62:43253/ using document root <none> and password file <none>
I20260807 18:20:25.281966 17259 fs_manager.cc:359] Metadata directory not provided
I20260807 18:20:25.282099 17259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:20:25.282399 17259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:20:25.286031 17259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "6d5fe81de4de48bf934b00a503a949c3"
format_stamp: "Formatted at 2026-08-07 18:20:25 on dist-test-slave-39l8"
server_key: "22d22ca098878046a5e7aa7e84baca48"
server_key_iv: "6d938dd0017aa0a2a661fe858446bdba"
server_key_version: "encryptionkey@0"
I20260807 18:20:25.287074 17259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "6d5fe81de4de48bf934b00a503a949c3"
format_stamp: "Formatted at 2026-08-07 18:20:25 on dist-test-slave-39l8"
server_key: "22d22ca098878046a5e7aa7e84baca48"
server_key_iv: "6d938dd0017aa0a2a661fe858446bdba"
server_key_version: "encryptionkey@0"
I20260807 18:20:25.293083 17259 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 18:20:25.297359 17279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:25.298985 17259 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 18:20:25.299259 17259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "6d5fe81de4de48bf934b00a503a949c3"
format_stamp: "Formatted at 2026-08-07 18:20:25 on dist-test-slave-39l8"
server_key: "22d22ca098878046a5e7aa7e84baca48"
server_key_iv: "6d938dd0017aa0a2a661fe858446bdba"
server_key_version: "encryptionkey@0"
I20260807 18:20:25.299494 17259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:20:25.306216 17259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:20:25.311300 17259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:20:25.311666 17259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:20:25.332638 17259 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:43205
I20260807 18:20:25.332697 17351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:43205 every 8 connection(s)
I20260807 18:20:25.334219 17259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:20:25.338886 17352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:20:25.342557 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 17259
I20260807 18:20:25.342844 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:20:25.343432 17848 external_mini_cluster.cc:1428] Setting key 08f8068ab2adaa6c8fcd8054ae90e062
I20260807 18:20:25.352672 17356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:25.345727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36901 (local address 127.17.110.62:43205)
0807 18:20:25.346690 (+ 963us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:25.346763 (+ 73us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:25.346826 (+ 63us) server_negotiation.cc:407] Connection header received
0807 18:20:25.348700 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:25.348729 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:25.349149 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:25.349628 (+ 479us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:25.350883 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:25.350890 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:25.350904 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:20:25.350920 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:25.350979 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:20:25.350999 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:25.351265 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:25.351425 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 18:20:25.351637 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":685,"thread_start_us":404,"threads_started":1}
I20260807 18:20:25.352895 17352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d5fe81de4de48bf934b00a503a949c3: Bootstrap starting.
I20260807 18:20:25.356915 17352 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d5fe81de4de48bf934b00a503a949c3: Neither blocks nor log segments found. Creating new log.
I20260807 18:20:25.357098 17848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:20:25.358160 17352 log.cc:824] T 00000000000000000000000000000000 P 6d5fe81de4de48bf934b00a503a949c3: Log is configured to *not* fsync() on all Append() calls
I20260807 18:20:25.361826 17352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d5fe81de4de48bf934b00a503a949c3: No bootstrap required, opened a new log
I20260807 18:20:25.365437 17356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:25.362448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38265 (local address 127.17.110.62:43205)
0807 18:20:25.362683 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:25.362690 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:25.363173 (+ 483us) server_negotiation.cc:407] Connection header received
0807 18:20:25.363334 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:25.363339 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:25.363414 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:25.363556 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:25.364636 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:25.364643 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:25.364647 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:20:25.364669 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:25.364724 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:20:25.364750 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:25.364887 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:25.365005 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 18:20:25.365071 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260807 18:20:25.372263 17352 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d5fe81de4de48bf934b00a503a949c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d5fe81de4de48bf934b00a503a949c3" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 43205 } }
I20260807 18:20:25.372706 17352 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d5fe81de4de48bf934b00a503a949c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:20:25.372788 17352 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d5fe81de4de48bf934b00a503a949c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d5fe81de4de48bf934b00a503a949c3, State: Initialized, Role: FOLLOWER
I20260807 18:20:25.373479 17352 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d5fe81de4de48bf934b00a503a949c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d5fe81de4de48bf934b00a503a949c3" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 43205 } }
I20260807 18:20:25.373822 17352 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d5fe81de4de48bf934b00a503a949c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:20:25.374027 17352 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d5fe81de4de48bf934b00a503a949c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:20:25.374236 17352 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d5fe81de4de48bf934b00a503a949c3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:20:25.376847 17352 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d5fe81de4de48bf934b00a503a949c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d5fe81de4de48bf934b00a503a949c3" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 43205 } }
I20260807 18:20:25.377485 17352 leader_election.cc:302] T 00000000000000000000000000000000 P 6d5fe81de4de48bf934b00a503a949c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d5fe81de4de48bf934b00a503a949c3; no voters:
I20260807 18:20:25.378161 17352 leader_election.cc:288] T 00000000000000000000000000000000 P 6d5fe81de4de48bf934b00a503a949c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:20:25.378363 17365 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d5fe81de4de48bf934b00a503a949c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:20:25.379180 17365 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d5fe81de4de48bf934b00a503a949c3 [term 1 LEADER]: Becoming Leader. State: Replica: 6d5fe81de4de48bf934b00a503a949c3, State: Running, Role: LEADER
I20260807 18:20:25.379873 17365 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d5fe81de4de48bf934b00a503a949c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d5fe81de4de48bf934b00a503a949c3" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 43205 } }
I20260807 18:20:25.380635 17352 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d5fe81de4de48bf934b00a503a949c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:20:25.382958 17370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d5fe81de4de48bf934b00a503a949c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d5fe81de4de48bf934b00a503a949c3. Latest consensus state: current_term: 1 leader_uuid: "6d5fe81de4de48bf934b00a503a949c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d5fe81de4de48bf934b00a503a949c3" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 43205 } } }
I20260807 18:20:25.383484 17370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d5fe81de4de48bf934b00a503a949c3 [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:25.383384 17369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d5fe81de4de48bf934b00a503a949c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d5fe81de4de48bf934b00a503a949c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d5fe81de4de48bf934b00a503a949c3" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 43205 } } }
I20260807 18:20:25.383687 17369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d5fe81de4de48bf934b00a503a949c3 [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:25.386304 17373 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:20:25.392170 17373 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:20:25.402281 17373 catalog_manager.cc:1347] Generated new cluster ID: 8c20f05e9e1a428da0a72722653eed84
I20260807 18:20:25.402485 17373 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:20:25.441761 17373 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:20:25.442463 17373 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:20:25.449134 17373 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d5fe81de4de48bf934b00a503a949c3: Generated new TSK 0
I20260807 18:20:25.450116 17373 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:20:25.464610 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 17259
W20260807 18:20:25.466351 17346 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36901
W20260807 18:20:25.467228 17848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 17259: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:20:25.467396 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 17259
2026-08-07T18:20:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2408 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:20:25 dist-test-slave-39l8 krb5kdc[17393](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:20:25 dist-test-slave-39l8 krb5kdc[17393](info): set up 2 sockets
Aug 07 18:20:25 dist-test-slave-39l8 krb5kdc[17393](info): commencing operation
krb5kdc: starting...
W20260807 18:20:26.821009 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.305s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:20:27 dist-test-slave-39l8 krb5kdc[17393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:20:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:20:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.110.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
I20260807 18:20:27.219121 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:40713
--webserver_interface=127.17.110.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:45701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:40713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:20:27.624322 17411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:20:27.624750 17411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:20:27.624924 17411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:20:27.632848 17411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:20:27.633035 17411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:20:27.633165 17411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:20:27.633283 17411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:20:27.633363 17411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:20:27.644425 17411 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:45701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:40713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:40713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:20:27.645920 17411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:20:27.647477 17411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:20:27.662266 17417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:27.662331 17420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:27.662354 17418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:20:27.664836 17411 server_base.cc:1034] running on GCE node
I20260807 18:20:27.665814 17411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:20:27.667511 17411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:20:27.668783 17411 hybrid_clock.cc:648] HybridClock initialized: now 1786126827668742 us; error 44 us; skew 500 ppm
I20260807 18:20:27.669191 17411 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:20:27 dist-test-slave-39l8 krb5kdc[17393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126827, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:20:27.679382 17411 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:20:27.681375 17411 webserver.cc:460] Webserver started at http://127.17.110.62:41155/ using document root <none> and password file <none>
I20260807 18:20:27.682201 17411 fs_manager.cc:359] Metadata directory not provided
I20260807 18:20:27.682365 17411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:20:27.682789 17411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:20:27.686458 17411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "48ff520730fc44e08ba7623e8d4d23f8"
format_stamp: "Formatted at 2026-08-07 18:20:27 on dist-test-slave-39l8"
server_key: "a6681591efa26a7c15978c384b1129f9"
server_key_iv: "c383d932259aa47219ee5df63b0a79d6"
server_key_version: "encryptionkey@0"
I20260807 18:20:27.687516 17411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "48ff520730fc44e08ba7623e8d4d23f8"
format_stamp: "Formatted at 2026-08-07 18:20:27 on dist-test-slave-39l8"
server_key: "a6681591efa26a7c15978c384b1129f9"
server_key_iv: "c383d932259aa47219ee5df63b0a79d6"
server_key_version: "encryptionkey@0"
I20260807 18:20:27.694233 17411 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.004s
I20260807 18:20:27.698648 17432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:27.700393 17411 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:20:27.700639 17411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "48ff520730fc44e08ba7623e8d4d23f8"
format_stamp: "Formatted at 2026-08-07 18:20:27 on dist-test-slave-39l8"
server_key: "a6681591efa26a7c15978c384b1129f9"
server_key_iv: "c383d932259aa47219ee5df63b0a79d6"
server_key_version: "encryptionkey@0"
I20260807 18:20:27.700888 17411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:20:27.728296 17411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:20:27.733703 17411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:20:27.734061 17411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:20:27.755609 17411 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:40713
I20260807 18:20:27.755626 17498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:40713 every 8 connection(s)
I20260807 18:20:27.757154 17411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:20:27.759866 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 17411
I20260807 18:20:27.760082 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:20:27.760706 17848 external_mini_cluster.cc:1428] Setting key 8c423fbbc58840563fbda612613b03d3
I20260807 18:20:27.761768 17500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 18:20:27 dist-test-slave-39l8 krb5kdc[17393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:20:27.775134 17500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48ff520730fc44e08ba7623e8d4d23f8: Bootstrap starting.
I20260807 18:20:27.778923 17500 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48ff520730fc44e08ba7623e8d4d23f8: Neither blocks nor log segments found. Creating new log.
I20260807 18:20:27.780109 17500 log.cc:824] T 00000000000000000000000000000000 P 48ff520730fc44e08ba7623e8d4d23f8: Log is configured to *not* fsync() on all Append() calls
I20260807 18:20:27.784096 17500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48ff520730fc44e08ba7623e8d4d23f8: No bootstrap required, opened a new log
I20260807 18:20:27.785274 17505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:27.762682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34887 (local address 127.17.110.62:40713)
0807 18:20:27.763511 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:27.763524 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:27.763565 (+ 41us) server_negotiation.cc:407] Connection header received
0807 18:20:27.765206 (+ 1641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:27.765230 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:27.765660 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:27.766098 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:27.767786 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:27.769208 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:27.770532 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:27.771354 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:27.771577 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:27.776424 (+ 4847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:27.776434 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:27.776450 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:27.776505 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:27.779949 (+ 3444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:27.780917 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:27.780925 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:27.780932 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:27.781134 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:27.781681 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:27.781687 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:27.781690 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:27.782513 (+ 823us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:27.782868 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:27.783356 (+ 488us) server_negotiation.cc:299] Negotiation successful
0807 18:20:27.783840 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":355,"threads_started":1}
I20260807 18:20:27.789914 17848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:20:27.793805 17500 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48ff520730fc44e08ba7623e8d4d23f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ff520730fc44e08ba7623e8d4d23f8" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40713 } }
I20260807 18:20:27.794350 17500 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48ff520730fc44e08ba7623e8d4d23f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:20:27.794471 17500 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48ff520730fc44e08ba7623e8d4d23f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48ff520730fc44e08ba7623e8d4d23f8, State: Initialized, Role: FOLLOWER
I20260807 18:20:27.795325 17500 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48ff520730fc44e08ba7623e8d4d23f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ff520730fc44e08ba7623e8d4d23f8" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40713 } }
I20260807 18:20:27.795593 17500 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48ff520730fc44e08ba7623e8d4d23f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:20:27.795814 17500 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48ff520730fc44e08ba7623e8d4d23f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:20:27.796022 17500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48ff520730fc44e08ba7623e8d4d23f8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:20:27.798941 17500 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48ff520730fc44e08ba7623e8d4d23f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ff520730fc44e08ba7623e8d4d23f8" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40713 } }
I20260807 18:20:27.799578 17500 leader_election.cc:302] T 00000000000000000000000000000000 P 48ff520730fc44e08ba7623e8d4d23f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48ff520730fc44e08ba7623e8d4d23f8; no voters:
I20260807 18:20:27.800257 17500 leader_election.cc:288] T 00000000000000000000000000000000 P 48ff520730fc44e08ba7623e8d4d23f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:20:27.800475 17514 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48ff520730fc44e08ba7623e8d4d23f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:20:27.801836 17514 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48ff520730fc44e08ba7623e8d4d23f8 [term 1 LEADER]: Becoming Leader. State: Replica: 48ff520730fc44e08ba7623e8d4d23f8, State: Running, Role: LEADER
I20260807 18:20:27.802551 17514 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48ff520730fc44e08ba7623e8d4d23f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ff520730fc44e08ba7623e8d4d23f8" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40713 } }
I20260807 18:20:27.803041 17500 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48ff520730fc44e08ba7623e8d4d23f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:20:27.805666 17517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48ff520730fc44e08ba7623e8d4d23f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48ff520730fc44e08ba7623e8d4d23f8. Latest consensus state: current_term: 1 leader_uuid: "48ff520730fc44e08ba7623e8d4d23f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ff520730fc44e08ba7623e8d4d23f8" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40713 } } }
I20260807 18:20:27.805889 17515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48ff520730fc44e08ba7623e8d4d23f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48ff520730fc44e08ba7623e8d4d23f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ff520730fc44e08ba7623e8d4d23f8" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40713 } } }
I20260807 18:20:27.806119 17517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48ff520730fc44e08ba7623e8d4d23f8 [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:27.806190 17515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48ff520730fc44e08ba7623e8d4d23f8 [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:27.816335 17505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:27.794961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54721 (local address 127.17.110.62:40713)
0807 18:20:27.795152 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:27.795161 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:27.795695 (+ 534us) server_negotiation.cc:407] Connection header received
0807 18:20:27.795907 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:27.795912 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:27.796028 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:27.796182 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:27.797676 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:27.798809 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:27.800384 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:27.801194 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:27.802108 (+ 914us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:27.804873 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:27.804882 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:27.804890 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:27.804950 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:27.810290 (+ 5340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:27.811649 (+ 1359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:27.811659 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:27.811665 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:27.811996 (+ 331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:27.812619 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:27.812627 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:27.812633 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:27.813162 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:27.813561 (+ 399us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:27.815401 (+ 1840us) server_negotiation.cc:299] Negotiation successful
0807 18:20:27.815768 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 18:20:27.819062 17524 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:20:27.825402 17524 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:20:27.835649 17524 catalog_manager.cc:1347] Generated new cluster ID: af9dc9bfe6bb458893f6469b635e1c6b
I20260807 18:20:27.835810 17524 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:20:27.854792 17524 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:20:27.856333 17524 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:20:27.867933 17524 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48ff520730fc44e08ba7623e8d4d23f8: Generated new TSK 0
I20260807 18:20:27.869320 17524 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:20:27.888845 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 17411
I20260807 18:20:27.890393 17494 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:20:27.989979 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 17411
2026-08-07T18:20:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2523 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:20:28 dist-test-slave-39l8 krb5kdc[17544](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:20:28 dist-test-slave-39l8 krb5kdc[17544](info): set up 2 sockets
Aug 07 18:20:28 dist-test-slave-39l8 krb5kdc[17544](info): commencing operation
krb5kdc: starting...
W20260807 18:20:29.383634 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.344s user 0.006s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:20:29 dist-test-slave-39l8 krb5kdc[17544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:20:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:20:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.110.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
I20260807 18:20:29.750029 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:33973
--webserver_interface=127.17.110.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:40319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:33973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:20:30.121912 17562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:20:30.122253 17562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:20:30.122359 17562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:20:30.129959 17562 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:20:30.130108 17562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:20:30.130158 17562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:20:30.130234 17562 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:20:30.130280 17562 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:20:30.140918 17562 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:40319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:33973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:33973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:20:30.142274 17562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:20:30.143697 17562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:20:30.156395 17569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:30.156441 17573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:30.156390 17570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:20:30.158025 17562 server_base.cc:1034] running on GCE node
I20260807 18:20:30.159073 17562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:20:30.160733 17562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:20:30.161995 17562 hybrid_clock.cc:648] HybridClock initialized: now 1786126830161959 us; error 54 us; skew 500 ppm
I20260807 18:20:30.162397 17562 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:20:30 dist-test-slave-39l8 krb5kdc[17544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126830, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:20:30.165927 17562 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:20:30.167775 17562 webserver.cc:460] Webserver started at http://127.17.110.62:45813/ using document root <none> and password file <none>
I20260807 18:20:30.168597 17562 fs_manager.cc:359] Metadata directory not provided
I20260807 18:20:30.168777 17562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:20:30.169131 17562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:20:30.172588 17562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "116f24c6ce5149e0acbdf77d8ae724fb"
format_stamp: "Formatted at 2026-08-07 18:20:30 on dist-test-slave-39l8"
server_key: "fe8a6ab878168feb6184a46ea4ae5e91"
server_key_iv: "ad5aab8074531259edf53565dfe8b03e"
server_key_version: "encryptionkey@0"
I20260807 18:20:30.173497 17562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "116f24c6ce5149e0acbdf77d8ae724fb"
format_stamp: "Formatted at 2026-08-07 18:20:30 on dist-test-slave-39l8"
server_key: "fe8a6ab878168feb6184a46ea4ae5e91"
server_key_iv: "ad5aab8074531259edf53565dfe8b03e"
server_key_version: "encryptionkey@0"
I20260807 18:20:30.179421 17562 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 18:20:30.183640 17582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:30.185366 17562 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:20:30.185734 17562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "116f24c6ce5149e0acbdf77d8ae724fb"
format_stamp: "Formatted at 2026-08-07 18:20:30 on dist-test-slave-39l8"
server_key: "fe8a6ab878168feb6184a46ea4ae5e91"
server_key_iv: "ad5aab8074531259edf53565dfe8b03e"
server_key_version: "encryptionkey@0"
I20260807 18:20:30.186007 17562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:20:30.206581 17562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:20:30.211557 17562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:20:30.211899 17562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:20:30.232820 17562 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:33973
I20260807 18:20:30.232841 17659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:33973 every 8 connection(s)
I20260807 18:20:30.234421 17562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:20:30.236538 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 17562
I20260807 18:20:30.236783 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:20:30.237377 17848 external_mini_cluster.cc:1428] Setting key d4a04092523ca5c14bae8e448e8474bb
I20260807 18:20:30.239670 17660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:20:30.251658 17663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:30.239385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54785 (local address 127.17.110.62:33973)
0807 18:20:30.240378 (+ 993us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:30.240391 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:30.240434 (+ 43us) server_negotiation.cc:407] Connection header received
0807 18:20:30.242185 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:30.242211 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:30.242672 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:30.243225 (+ 553us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:30.243827 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:30.245280 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:30.246694 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:30.247463 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:30.247674 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:30.249009 (+ 1335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:30.249018 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:30.249038 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:20:30.249688 (+ 650us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:30.249781 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:20:30.249809 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:30.249966 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:30.250224 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 18:20:30.250524 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":762,"thread_start_us":443,"threads_started":1}
I20260807 18:20:30.252077 17660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 116f24c6ce5149e0acbdf77d8ae724fb: Bootstrap starting.
I20260807 18:20:30.255685 17660 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 116f24c6ce5149e0acbdf77d8ae724fb: Neither blocks nor log segments found. Creating new log.
I20260807 18:20:30.255730 17848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:20:30.256794 17660 log.cc:824] T 00000000000000000000000000000000 P 116f24c6ce5149e0acbdf77d8ae724fb: Log is configured to *not* fsync() on all Append() calls
I20260807 18:20:30.260258 17660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 116f24c6ce5149e0acbdf77d8ae724fb: No bootstrap required, opened a new log
I20260807 18:20:30.267917 17663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:30.260312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38421 (local address 127.17.110.62:33973)
0807 18:20:30.260521 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:30.260530 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:30.261090 (+ 560us) server_negotiation.cc:407] Connection header received
0807 18:20:30.261351 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:30.261356 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:30.261435 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:30.261531 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:30.262269 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:30.263131 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:30.265143 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:30.265635 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:30.265748 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:30.267093 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:30.267102 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:30.267105 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:20:30.267127 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:30.267183 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:20:30.267200 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:30.267277 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:30.267453 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 18:20:30.267541 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 18:20:30.270962 17660 raft_consensus.cc:348] T 00000000000000000000000000000000 P 116f24c6ce5149e0acbdf77d8ae724fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "116f24c6ce5149e0acbdf77d8ae724fb" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 33973 } }
I20260807 18:20:30.271445 17660 raft_consensus.cc:374] T 00000000000000000000000000000000 P 116f24c6ce5149e0acbdf77d8ae724fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:20:30.271564 17660 raft_consensus.cc:729] T 00000000000000000000000000000000 P 116f24c6ce5149e0acbdf77d8ae724fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116f24c6ce5149e0acbdf77d8ae724fb, State: Initialized, Role: FOLLOWER
I20260807 18:20:30.272320 17660 consensus_queue.cc:260] T 00000000000000000000000000000000 P 116f24c6ce5149e0acbdf77d8ae724fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "116f24c6ce5149e0acbdf77d8ae724fb" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 33973 } }
I20260807 18:20:30.272608 17660 raft_consensus.cc:388] T 00000000000000000000000000000000 P 116f24c6ce5149e0acbdf77d8ae724fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:20:30.272776 17660 raft_consensus.cc:482] T 00000000000000000000000000000000 P 116f24c6ce5149e0acbdf77d8ae724fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:20:30.273001 17660 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 116f24c6ce5149e0acbdf77d8ae724fb [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:20:30.275645 17660 raft_consensus.cc:504] T 00000000000000000000000000000000 P 116f24c6ce5149e0acbdf77d8ae724fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "116f24c6ce5149e0acbdf77d8ae724fb" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 33973 } }
I20260807 18:20:30.276432 17660 leader_election.cc:302] T 00000000000000000000000000000000 P 116f24c6ce5149e0acbdf77d8ae724fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 116f24c6ce5149e0acbdf77d8ae724fb; no voters:
I20260807 18:20:30.277209 17660 leader_election.cc:288] T 00000000000000000000000000000000 P 116f24c6ce5149e0acbdf77d8ae724fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:20:30.277377 17674 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 116f24c6ce5149e0acbdf77d8ae724fb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:20:30.278221 17674 raft_consensus.cc:686] T 00000000000000000000000000000000 P 116f24c6ce5149e0acbdf77d8ae724fb [term 1 LEADER]: Becoming Leader. State: Replica: 116f24c6ce5149e0acbdf77d8ae724fb, State: Running, Role: LEADER
I20260807 18:20:30.278879 17674 consensus_queue.cc:237] T 00000000000000000000000000000000 P 116f24c6ce5149e0acbdf77d8ae724fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "116f24c6ce5149e0acbdf77d8ae724fb" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 33973 } }
I20260807 18:20:30.279644 17660 sys_catalog.cc:564] T 00000000000000000000000000000000 P 116f24c6ce5149e0acbdf77d8ae724fb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:20:30.282155 17675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 116f24c6ce5149e0acbdf77d8ae724fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "116f24c6ce5149e0acbdf77d8ae724fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "116f24c6ce5149e0acbdf77d8ae724fb" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 33973 } } }
I20260807 18:20:30.282716 17675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 116f24c6ce5149e0acbdf77d8ae724fb [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:30.282929 17676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 116f24c6ce5149e0acbdf77d8ae724fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 116f24c6ce5149e0acbdf77d8ae724fb. Latest consensus state: current_term: 1 leader_uuid: "116f24c6ce5149e0acbdf77d8ae724fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "116f24c6ce5149e0acbdf77d8ae724fb" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 33973 } } }
I20260807 18:20:30.283270 17676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 116f24c6ce5149e0acbdf77d8ae724fb [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:30.287228 17685 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:20:30.292796 17685 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:20:30.302811 17685 catalog_manager.cc:1347] Generated new cluster ID: cdbea483499a42609bf73937716af1d3
I20260807 18:20:30.302955 17685 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:20:30.317957 17685 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:20:30.319504 17685 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:20:30.330763 17685 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 116f24c6ce5149e0acbdf77d8ae724fb: Generated new TSK 0
I20260807 18:20:30.331739 17685 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:20:30.344678 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 17562
W20260807 18:20:30.346010 17655 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54785
W20260807 18:20:30.346860 17848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 17562: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:20:30.347023 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 17562
2026-08-07T18:20:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2353 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:20:30 dist-test-slave-39l8 krb5kdc[17708](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:20:30 dist-test-slave-39l8 krb5kdc[17708](info): set up 2 sockets
Aug 07 18:20:30 dist-test-slave-39l8 krb5kdc[17708](info): commencing operation
krb5kdc: starting...
W20260807 18:20:31.762770 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.369s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:20:31 dist-test-slave-39l8 krb5kdc[17708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:20:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:20:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.110.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
I20260807 18:20:32.135336 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:39127
--webserver_interface=127.17.110.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:40319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:39127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:20:32.511976 17726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:20:32.512399 17726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:20:32.512581 17726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:20:32.520205 17726 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:20:32.520399 17726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:20:32.520519 17726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:20:32.520637 17726 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:20:32.520710 17726 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:20:32.531376 17726 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:40319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:39127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:39127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:20:32.532843 17726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:20:32.534276 17726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:20:32.547251 17733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:32.547298 17735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:32.547266 17732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:20:32.548472 17726 server_base.cc:1034] running on GCE node
I20260807 18:20:32.549333 17726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:20:32.550928 17726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:20:32.552165 17726 hybrid_clock.cc:648] HybridClock initialized: now 1786126832552110 us; error 40 us; skew 500 ppm
I20260807 18:20:32.552593 17726 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:20:32 dist-test-slave-39l8 krb5kdc[17708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126832, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:20:32.556186 17726 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:20:32.557808 17726 webserver.cc:460] Webserver started at http://127.17.110.62:43041/ using document root <none> and password file <none>
I20260807 18:20:32.558539 17726 fs_manager.cc:359] Metadata directory not provided
I20260807 18:20:32.558661 17726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:20:32.559046 17726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:20:32.562436 17726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "c1c3b954d2c1451694917069e31e447d"
format_stamp: "Formatted at 2026-08-07 18:20:32 on dist-test-slave-39l8"
server_key: "d059dbbe497ff1e4837444fb6ccaaf81"
server_key_iv: "8de40ff3a98bc8961772ae20870aa940"
server_key_version: "encryptionkey@0"
I20260807 18:20:32.563351 17726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "c1c3b954d2c1451694917069e31e447d"
format_stamp: "Formatted at 2026-08-07 18:20:32 on dist-test-slave-39l8"
server_key: "d059dbbe497ff1e4837444fb6ccaaf81"
server_key_iv: "8de40ff3a98bc8961772ae20870aa940"
server_key_version: "encryptionkey@0"
I20260807 18:20:32.569221 17726 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:20:32.573470 17744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:32.575130 17726 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:20:32.575350 17726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "c1c3b954d2c1451694917069e31e447d"
format_stamp: "Formatted at 2026-08-07 18:20:32 on dist-test-slave-39l8"
server_key: "d059dbbe497ff1e4837444fb6ccaaf81"
server_key_iv: "8de40ff3a98bc8961772ae20870aa940"
server_key_version: "encryptionkey@0"
I20260807 18:20:32.575562 17726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:20:32.582224 17726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:20:32.587193 17726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:20:32.587533 17726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:20:32.608466 17726 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:39127
I20260807 18:20:32.608510 17813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:39127 every 8 connection(s)
I20260807 18:20:32.610047 17726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:20:32.613149 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 17726
I20260807 18:20:32.613373 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:20:32.613848 17848 external_mini_cluster.cc:1428] Setting key fa73f1946355dbcea95e6ed146e085ab
I20260807 18:20:32.614912 17814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:20:32.624105 17817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:32.616021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52181 (local address 127.17.110.62:39127)
0807 18:20:32.617003 (+ 982us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:32.617022 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:32.617078 (+ 56us) server_negotiation.cc:407] Connection header received
0807 18:20:32.619228 (+ 2150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:32.619264 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:32.619864 (+ 600us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:32.620425 (+ 561us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:32.621652 (+ 1227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:32.621661 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:32.621681 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:20:32.621710 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:32.621816 (+ 106us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:20:32.621851 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:32.622121 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:32.622296 (+ 175us) server_negotiation.cc:299] Negotiation successful
0807 18:20:32.622550 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":387,"threads_started":1}
I20260807 18:20:32.628755 17848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:20:32.631547 17814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1c3b954d2c1451694917069e31e447d: Bootstrap starting.
I20260807 18:20:32.635674 17814 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1c3b954d2c1451694917069e31e447d: Neither blocks nor log segments found. Creating new log.
I20260807 18:20:32.636466 17817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:32.633640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46747 (local address 127.17.110.62:39127)
0807 18:20:32.633856 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:32.633862 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:32.634233 (+ 371us) server_negotiation.cc:407] Connection header received
0807 18:20:32.634383 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:32.634390 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:32.634461 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:32.634594 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:32.635700 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:32.635709 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:32.635715 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:20:32.635736 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:32.635791 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:20:32.635813 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:32.635960 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:32.636067 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 18:20:32.636140 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 18:20:32.637173 17814 log.cc:824] T 00000000000000000000000000000000 P c1c3b954d2c1451694917069e31e447d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:20:32.640408 17814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1c3b954d2c1451694917069e31e447d: No bootstrap required, opened a new log
I20260807 18:20:32.648851 17814 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1c3b954d2c1451694917069e31e447d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1c3b954d2c1451694917069e31e447d" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 39127 } }
I20260807 18:20:32.649330 17814 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1c3b954d2c1451694917069e31e447d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:20:32.649459 17814 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1c3b954d2c1451694917069e31e447d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1c3b954d2c1451694917069e31e447d, State: Initialized, Role: FOLLOWER
I20260807 18:20:32.650081 17814 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1c3b954d2c1451694917069e31e447d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1c3b954d2c1451694917069e31e447d" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 39127 } }
I20260807 18:20:32.650358 17814 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1c3b954d2c1451694917069e31e447d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:20:32.650518 17814 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1c3b954d2c1451694917069e31e447d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:20:32.650775 17814 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1c3b954d2c1451694917069e31e447d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:20:32.653512 17814 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1c3b954d2c1451694917069e31e447d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1c3b954d2c1451694917069e31e447d" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 39127 } }
I20260807 18:20:32.654173 17814 leader_election.cc:302] T 00000000000000000000000000000000 P c1c3b954d2c1451694917069e31e447d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1c3b954d2c1451694917069e31e447d; no voters:
I20260807 18:20:32.654907 17814 leader_election.cc:288] T 00000000000000000000000000000000 P c1c3b954d2c1451694917069e31e447d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:20:32.655016 17826 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1c3b954d2c1451694917069e31e447d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:20:32.655808 17826 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1c3b954d2c1451694917069e31e447d [term 1 LEADER]: Becoming Leader. State: Replica: c1c3b954d2c1451694917069e31e447d, State: Running, Role: LEADER
I20260807 18:20:32.656361 17826 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1c3b954d2c1451694917069e31e447d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1c3b954d2c1451694917069e31e447d" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 39127 } }
I20260807 18:20:32.657141 17814 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1c3b954d2c1451694917069e31e447d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:20:32.659377 17828 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1c3b954d2c1451694917069e31e447d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1c3b954d2c1451694917069e31e447d. Latest consensus state: current_term: 1 leader_uuid: "c1c3b954d2c1451694917069e31e447d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1c3b954d2c1451694917069e31e447d" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 39127 } } }
I20260807 18:20:32.659852 17828 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1c3b954d2c1451694917069e31e447d [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:32.659713 17827 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1c3b954d2c1451694917069e31e447d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1c3b954d2c1451694917069e31e447d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1c3b954d2c1451694917069e31e447d" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 39127 } } }
I20260807 18:20:32.659986 17827 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1c3b954d2c1451694917069e31e447d [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:32.661825 17836 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:20:32.667798 17836 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:20:32.678887 17836 catalog_manager.cc:1347] Generated new cluster ID: 083df31750a2443b87a0bc04c7155e89
I20260807 18:20:32.679091 17836 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:20:32.708900 17836 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:20:32.709409 17836 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:20:32.720795 17836 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1c3b954d2c1451694917069e31e447d: Generated new TSK 0
I20260807 18:20:32.721781 17836 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:20:32.736182 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 17726
W20260807 18:20:32.737640 17808 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:52181
W20260807 18:20:32.738590 17848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 17726: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:20:32.738838 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 17726
2026-08-07T18:20:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2402 ms)
[----------] 18 tests from AuthTokenIssuingTest (43365 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:20:32 dist-test-slave-39l8 krb5kdc[17865](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:20:32 dist-test-slave-39l8 krb5kdc[17865](info): set up 2 sockets
Aug 07 18:20:32 dist-test-slave-39l8 krb5kdc[17865](info): commencing operation
krb5kdc: starting...
W20260807 18:20:34.176285 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.379s user 0.003s sys 0.059s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:20:34 dist-test-slave-39l8 krb5kdc[17865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:20:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:20:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.110.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
I20260807 18:20:34.550803 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:37121
--webserver_interface=127.17.110.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:32781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:37121,127.17.110.61:45869,127.17.110.60:33067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:20:34.929098 17880 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:20:34.929611 17880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:20:34.929772 17880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:20:34.929930 17880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:20:34.938356 17880 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:20:34.938548 17880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:20:34.938663 17880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:20:34.938797 17880 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:20:34.938923 17880 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:20:34.950246 17880 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:32781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:37121,127.17.110.61:45869,127.17.110.60:33067
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:37121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:20:34.951856 17880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:20:34.953830 17880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:20:34.968768 17886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:34.968739 17888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:34.968808 17890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:20:34.969686 17880 server_base.cc:1034] running on GCE node
I20260807 18:20:34.970862 17880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:20:34.972648 17880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:20:34.974133 17880 hybrid_clock.cc:648] HybridClock initialized: now 1786126834974057 us; error 42 us; skew 500 ppm
I20260807 18:20:34.974941 17880 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:20:34 dist-test-slave-39l8 krb5kdc[17865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126834, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:20:34.979658 17880 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:20:34.981635 17880 webserver.cc:460] Webserver started at http://127.17.110.62:34981/ using document root <none> and password file <none>
I20260807 18:20:34.982548 17880 fs_manager.cc:359] Metadata directory not provided
I20260807 18:20:34.982785 17880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:20:34.983189 17880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:20:34.987178 17880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "b4fc9dfe61b34502a69682b4251e1a43"
format_stamp: "Formatted at 2026-08-07 18:20:34 on dist-test-slave-39l8"
server_key: "5a75b09a3efa57c9f36f87d00595523c"
server_key_iv: "39d75e773f3d9581e226c9d06741ff2b"
server_key_version: "encryptionkey@0"
I20260807 18:20:34.988224 17880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "b4fc9dfe61b34502a69682b4251e1a43"
format_stamp: "Formatted at 2026-08-07 18:20:34 on dist-test-slave-39l8"
server_key: "5a75b09a3efa57c9f36f87d00595523c"
server_key_iv: "39d75e773f3d9581e226c9d06741ff2b"
server_key_version: "encryptionkey@0"
I20260807 18:20:34.995718 17880 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260807 18:20:35.001016 17898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:35.003047 17880 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 18:20:35.003324 17880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "b4fc9dfe61b34502a69682b4251e1a43"
format_stamp: "Formatted at 2026-08-07 18:20:34 on dist-test-slave-39l8"
server_key: "5a75b09a3efa57c9f36f87d00595523c"
server_key_iv: "39d75e773f3d9581e226c9d06741ff2b"
server_key_version: "encryptionkey@0"
I20260807 18:20:35.003628 17880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:20:35.047883 17880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:20:35.053392 17880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:20:35.053755 17880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:20:35.076925 17880 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:37121
I20260807 18:20:35.076939 17976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:37121 every 8 connection(s)
I20260807 18:20:35.078619 17880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:20:35.081274 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 17880
I20260807 18:20:35.081719 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:20:35.082450 17848 external_mini_cluster.cc:1428] Setting key 705f9ab014d07de3d945adfa2fbf7816
I20260807 18:20:35.084004 17977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:20:35.092396 17977 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.110.62" port: 37121 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:20:35 dist-test-slave-39l8 krb5kdc[17865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126834, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.110.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.61@KRBTEST.COM" created.
I20260807 18:20:35.129822 17980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:35.096668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35298 (local address 127.17.110.62:37121)
0807 18:20:35.098267 (+ 1599us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:35.098280 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:35.099095 (+ 815us) server_negotiation.cc:407] Connection header received
0807 18:20:35.103006 (+ 3911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:35.103014 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:35.103514 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:35.103764 (+ 250us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:35.106437 (+ 2673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:35.107929 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:35.111546 (+ 3617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:35.112299 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:35.112482 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:35.118423 (+ 5941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:35.118445 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:35.118462 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:35.118511 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:35.122806 (+ 4295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:35.124568 (+ 1762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:35.124577 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:35.124580 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:35.124847 (+ 267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:35.125552 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:35.125557 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:35.125559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:35.126523 (+ 964us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:35.126807 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:35.127662 (+ 855us) server_negotiation.cc:299] Negotiation successful
0807 18:20:35.128040 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1349,"spinlock_wait_cycles":47360,"thread_start_us":934,"threads_started":1}
I20260807 18:20:35.129843 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:35.096671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:37121 (local address 127.0.0.1:35298)
0807 18:20:35.098845 (+ 2174us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:35.098915 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0807 18:20:35.102012 (+ 3097us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:20:35.104001 (+ 1989us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:20:35.104066 (+ 65us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:20:35.105368 (+ 1302us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:20:35.106083 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:35.106123 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:35.108465 (+ 2342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:35.108474 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:35.111276 (+ 2802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:35.111310 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:35.112639 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:35.112650 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:35.112926 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:35.113889 (+ 963us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:20:35.113931 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:20:35.117802 (+ 3871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:20:35.123046 (+ 5244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:35.123142 (+ 96us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:35.123159 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:35.124217 (+ 1058us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:35.125041 (+ 824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:35.125048 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:35.125052 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:35.125353 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:35.126851 (+ 1498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:20:35.126859 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:20:35.127107 (+ 248us) client_negotiation.cc:769] Sending connection context
0807 18:20:35.127454 (+ 347us) client_negotiation.cc:240] Negotiation successful
0807 18:20:35.128232 (+ 778us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1837,"mutex_wait_us":213,"thread_start_us":1435,"threads_started":1}
I20260807 18:20:35.134037 17977 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.110.61" port: 45869 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:20:35.135831 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:35.134958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:45869 (local address 127.0.0.1:60300)
0807 18:20:35.135333 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:35.135609 (+ 276us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.61:45869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
W20260807 18:20:35.136354 17905 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.110.61:45869: connect: Connection refused (error 111)
I20260807 18:20:35.138859 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:35.138060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:45869 (local address 127.0.0.1:60306)
0807 18:20:35.138398 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:35.138497 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.61:45869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
W20260807 18:20:35.139181 17977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.61:45869: Network error: Client connection negotiation failed: client connection to 127.17.110.61:45869: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.61.keytab.
Entry for principal kudu/127.17.110.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.61.keytab.
I20260807 18:20:35.183326 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.61:45869
--webserver_interface=127.17.110.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.61.keytab
--principal=kudu/127.17.110.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:32781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:37121,127.17.110.61:45869,127.17.110.60:33067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:20:35.188673 17977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.61" port: 45869 } attempt: 1
I20260807 18:20:35.189983 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:35.189491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:45869 (local address 127.0.0.1:60318)
0807 18:20:35.189722 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:35.189840 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.61:45869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260807 18:20:35.191272 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:35.190847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:45869 (local address 127.0.0.1:60328)
0807 18:20:35.191005 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:35.191101 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.61:45869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 18:20:35.191471 17977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.61:45869: Network error: Client connection negotiation failed: client connection to 127.17.110.61:45869: connect: Connection refused (error 111)
I20260807 18:20:35.259811 17977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.61" port: 45869 } attempt: 2
I20260807 18:20:35.261368 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:35.260769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:45869 (local address 127.0.0.1:60334)
0807 18:20:35.261097 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:35.261216 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.61:45869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260807 18:20:35.262653 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:35.262213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:45869 (local address 127.0.0.1:60348)
0807 18:20:35.262390 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:35.262493 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.61:45869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260807 18:20:35.263056 17977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.61:45869: Network error: Client connection negotiation failed: client connection to 127.17.110.61:45869: connect: Connection refused (error 111)
I20260807 18:20:35.354614 17977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.61" port: 45869 } attempt: 3
I20260807 18:20:35.355983 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:35.355464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:45869 (local address 127.0.0.1:60350)
0807 18:20:35.355732 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:35.355833 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.61:45869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260807 18:20:35.357304 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:35.356852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:45869 (local address 127.0.0.1:60366)
0807 18:20:35.357029 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:35.357114 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.61:45869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260807 18:20:35.357555 17977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.61:45869: Network error: Client connection negotiation failed: client connection to 127.17.110.61:45869: connect: Connection refused (error 111)
I20260807 18:20:35.500838 17977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.61" port: 45869 } attempt: 4
I20260807 18:20:35.502219 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:35.501650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:45869 (local address 127.0.0.1:60378)
0807 18:20:35.501956 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:35.502055 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.61:45869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
I20260807 18:20:35.503439 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:35.502997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:45869 (local address 127.0.0.1:60392)
0807 18:20:35.503215 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:35.503293 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.61:45869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260807 18:20:35.503659 17977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.61:45869: Network error: Client connection negotiation failed: client connection to 127.17.110.61:45869: connect: Connection refused (error 111)
W20260807 18:20:35.601737 17984 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:20:35.602227 17984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:20:35.602398 17984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:20:35.602548 17984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:20:35.612149 17984 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:20:35.612330 17984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:20:35.612470 17984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:20:35.612565 17984 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:20:35.612694 17984 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:20:35.623831 17984 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:32781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:37121,127.17.110.61:45869,127.17.110.60:33067
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.61.keytab
--principal=kudu/127.17.110.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.61:45869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.110.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:20:35.625527 17984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:20:35.627071 17984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:20:35.645269 17989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:35.645228 17990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:35.645183 17993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:20:35.645190 17984 server_base.cc:1034] running on GCE node
I20260807 18:20:35.646859 17984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:20:35.648537 17984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:20:35.649868 17984 hybrid_clock.cc:648] HybridClock initialized: now 1786126835649772 us; error 51 us; skew 500 ppm
I20260807 18:20:35.650321 17984 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:20:35 dist-test-slave-39l8 krb5kdc[17865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126835, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:20:35.654338 17984 init.cc:377] Logged in from keytab as kudu/127.17.110.61@KRBTEST.COM (short username kudu)
I20260807 18:20:35.656414 17984 webserver.cc:460] Webserver started at http://127.17.110.61:42443/ using document root <none> and password file <none>
I20260807 18:20:35.657408 17984 fs_manager.cc:359] Metadata directory not provided
I20260807 18:20:35.657579 17984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:20:35.657974 17984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:20:35.661672 17984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-1/data/instance:
uuid: "69dbc49add32419e8509237bda6639d2"
format_stamp: "Formatted at 2026-08-07 18:20:35 on dist-test-slave-39l8"
server_key: "24d6d89b61855836683e8f79ac074a0e"
server_key_iv: "e81a81160893d3840e71271afa6a6ed7"
server_key_version: "encryptionkey@0"
I20260807 18:20:35.662681 17984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-1/wal/instance:
uuid: "69dbc49add32419e8509237bda6639d2"
format_stamp: "Formatted at 2026-08-07 18:20:35 on dist-test-slave-39l8"
server_key: "24d6d89b61855836683e8f79ac074a0e"
server_key_iv: "e81a81160893d3840e71271afa6a6ed7"
server_key_version: "encryptionkey@0"
I20260807 18:20:35.669382 17984 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 18:20:35.673933 18004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:35.675813 17984 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 18:20:35.676074 17984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-1/wal
uuid: "69dbc49add32419e8509237bda6639d2"
format_stamp: "Formatted at 2026-08-07 18:20:35 on dist-test-slave-39l8"
server_key: "24d6d89b61855836683e8f79ac074a0e"
server_key_iv: "e81a81160893d3840e71271afa6a6ed7"
server_key_version: "encryptionkey@0"
I20260807 18:20:35.676363 17984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:20:35.692464 17984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:20:35.698261 17984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:20:35.698544 17984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:20:35.724362 17984 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.61:45869
I20260807 18:20:35.724398 18077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.61:45869 every 8 connection(s)
I20260807 18:20:35.726094 17984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-1/data/info.pb
I20260807 18:20:35.727576 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 17984
I20260807 18:20:35.727821 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-1/wal/instance
I20260807 18:20:35.728308 17848 external_mini_cluster.cc:1428] Setting key 0efcf2b14baf721c4214a553862d6024
I20260807 18:20:35.731231 18078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:20:35.739099 18078 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.110.62" port: 37121 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:20:35 dist-test-slave-39l8 krb5kdc[17865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126835, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.61@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.110.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.60@KRBTEST.COM" created.
I20260807 18:20:35.767810 18082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:35.742509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35308 (local address 127.17.110.62:37121)
0807 18:20:35.743204 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:35.743214 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:35.744365 (+ 1151us) server_negotiation.cc:407] Connection header received
0807 18:20:35.747337 (+ 2972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:35.747346 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:35.747475 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:35.747647 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:35.749716 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:35.751075 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:35.752939 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:35.753736 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:35.753897 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:35.758435 (+ 4538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:35.758446 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:35.758454 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:35.758537 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:35.763440 (+ 4903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:35.764621 (+ 1181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:35.764631 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:35.764636 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:35.764817 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:35.765511 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:35.765520 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:35.765526 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:35.765893 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:35.766040 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:35.767073 (+ 1033us) server_negotiation.cc:299] Negotiation successful
0807 18:20:35.767331 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":356,"threads_started":1}
I20260807 18:20:35.769352 18088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:35.743160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:37121 (local address 127.0.0.1:35308)
0807 18:20:35.744122 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:35.744189 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 18:20:35.746833 (+ 2644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:20:35.747983 (+ 1150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:20:35.747994 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:20:35.748685 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:20:35.749379 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:35.749400 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:35.751298 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:35.751303 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:35.752732 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:35.752751 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:35.754001 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:35.754007 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:35.754251 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:35.755107 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:20:35.755135 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:20:35.758078 (+ 2943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:20:35.763815 (+ 5737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:35.763829 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:35.763846 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:35.764434 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:35.765000 (+ 566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:35.765004 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:35.765007 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:35.765296 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:35.766103 (+ 807us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:20:35.766111 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:20:35.766514 (+ 403us) client_negotiation.cc:769] Sending connection context
0807 18:20:35.766916 (+ 402us) client_negotiation.cc:240] Negotiation successful
0807 18:20:35.767414 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":426,"threads_started":1}
I20260807 18:20:35.772037 18078 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.110.61" port: 45869 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:20:35 dist-test-slave-39l8 krb5kdc[17865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126835, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.61@KRBTEST.COM for kudu/127.17.110.61@KRBTEST.COM
I20260807 18:20:35.793072 18088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:35.772928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:45869 (local address 127.0.0.1:60398)
0807 18:20:35.773230 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:35.773258 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 18:20:35.773407 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:20:35.775083 (+ 1676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:20:35.775087 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:20:35.775644 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:20:35.775819 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:35.775830 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:35.778020 (+ 2190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:35.778024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:35.779029 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:35.779045 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:35.780644 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:35.780654 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:35.780826 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:35.781673 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:20:35.781713 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:20:35.785311 (+ 3598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:20:35.790127 (+ 4816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:35.790133 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:35.790137 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:35.790558 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:35.791048 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:35.791056 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:35.791060 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:35.791215 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:35.791945 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:20:35.791950 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:20:35.792110 (+ 160us) client_negotiation.cc:769] Sending connection context
0807 18:20:35.792280 (+ 170us) client_negotiation.cc:240] Negotiation successful
0807 18:20:35.792654 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133}
I20260807 18:20:35.793211 18090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:35.773402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60398 (local address 127.17.110.61:45869)
0807 18:20:35.774280 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:35.774292 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:35.774334 (+ 42us) server_negotiation.cc:407] Connection header received
0807 18:20:35.774420 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:35.774432 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:35.774814 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:35.775029 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:35.776032 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:35.777820 (+ 1788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:35.779199 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:35.780311 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:35.780512 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:35.785952 (+ 5440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:35.785972 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:35.785990 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:35.786034 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:35.789913 (+ 3879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:35.790779 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:35.790787 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:35.790789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:35.790919 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:35.791365 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:35.791371 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:35.791374 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:35.791779 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:35.792016 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:35.792439 (+ 423us) server_negotiation.cc:299] Negotiation successful
0807 18:20:35.792798 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":80,"server-negotiator.queue_time_us":624,"thread_start_us":443,"threads_started":1}
I20260807 18:20:35.796054 18078 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.110.60" port: 33067 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:20:35.797747 18088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:35.797123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:33067 (local address 127.0.0.1:54128)
0807 18:20:35.797366 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:35.797544 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:33067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
W20260807 18:20:35.798169 18008 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.110.60:33067: connect: Connection refused (error 111)
I20260807 18:20:35.800324 18088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:35.799785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:33067 (local address 127.0.0.1:54144)
0807 18:20:35.799962 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:35.800182 (+ 220us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:33067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 18:20:35.800673 18078 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.60:33067: Network error: Client connection negotiation failed: client connection to 127.17.110.60:33067: connect: Connection refused (error 111)
I20260807 18:20:35.802904 17977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.61" port: 45869 } attempt: 5
Aug 07 18:20:35 dist-test-slave-39l8 krb5kdc[17865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126834, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for kudu/127.17.110.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.60.keytab.
Entry for principal kudu/127.17.110.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.60.keytab.
I20260807 18:20:35.815857 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.60:33067
--webserver_interface=127.17.110.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.60.keytab
--principal=kudu/127.17.110.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:32781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:37121,127.17.110.61:45869,127.17.110.60:33067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:20:35.820677 18090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:35.803837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60412 (local address 127.17.110.61:45869)
0807 18:20:35.804068 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:35.804076 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:35.804103 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:20:35.804181 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:35.804186 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:35.804277 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:35.804416 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:35.805388 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:35.806173 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:35.807690 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:35.808264 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:35.808397 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:35.813310 (+ 4913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:35.813354 (+ 44us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:35.813369 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:35.813434 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:35.817090 (+ 3656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:35.818263 (+ 1173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:35.818268 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:35.818271 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:35.818426 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:35.819033 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:35.819037 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:35.819040 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:35.819428 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:35.819567 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:35.820196 (+ 629us) server_negotiation.cc:299] Negotiation successful
0807 18:20:35.820419 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 18:20:35.820920 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:35.803728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:45869 (local address 127.0.0.1:60412)
0807 18:20:35.803932 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:35.803955 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:20:35.804087 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:20:35.804450 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:20:35.804453 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:20:35.804892 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:20:35.805110 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:35.805120 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:35.806360 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:35.806365 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:35.807421 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:35.807437 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:35.808560 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:35.808566 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:35.808771 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:35.810004 (+ 1233us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:20:35.810025 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:20:35.812995 (+ 2970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:20:35.817322 (+ 4327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:35.817332 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:35.817339 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:35.818070 (+ 731us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:35.818563 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:35.818571 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:35.818577 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:35.818882 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:35.819575 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:20:35.819582 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:20:35.819807 (+ 225us) client_negotiation.cc:769] Sending connection context
0807 18:20:35.819980 (+ 173us) client_negotiation.cc:240] Negotiation successful
0807 18:20:35.820429 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 18:20:35.822434 17977 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.110.60" port: 33067 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:20:35.823962 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:35.823374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:33067 (local address 127.0.0.1:54154)
0807 18:20:35.823606 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:35.823730 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:33067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 18:20:35.825256 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:35.824744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:33067 (local address 127.0.0.1:54156)
0807 18:20:35.824967 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:35.825087 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:33067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260807 18:20:35.825585 17977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.60:33067: Network error: Client connection negotiation failed: client connection to 127.17.110.60:33067: connect: Connection refused (error 111)
I20260807 18:20:35.850107 18078 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.60" port: 33067 } attempt: 1
I20260807 18:20:35.851676 18088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:35.851044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:33067 (local address 127.0.0.1:54168)
0807 18:20:35.851289 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:35.851415 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:33067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260807 18:20:35.852963 18088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:35.852575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:33067 (local address 127.0.0.1:54184)
0807 18:20:35.852734 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:35.852840 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:33067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 18:20:35.853250 18078 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.60:33067: Network error: Client connection negotiation failed: client connection to 127.17.110.60:33067: connect: Connection refused (error 111)
I20260807 18:20:35.876932 17977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.60" port: 33067 } attempt: 1
I20260807 18:20:35.878440 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:35.877722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:33067 (local address 127.0.0.1:54196)
0807 18:20:35.878014 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:35.878190 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:33067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260807 18:20:35.879756 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:35.879346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:33067 (local address 127.0.0.1:54202)
0807 18:20:35.879518 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:35.879617 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:33067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 18:20:35.880095 17977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.60:33067: Network error: Client connection negotiation failed: client connection to 127.17.110.60:33067: connect: Connection refused (error 111)
I20260807 18:20:35.921600 18078 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.60" port: 33067 } attempt: 2
I20260807 18:20:35.922989 18088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:35.922410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:33067 (local address 127.0.0.1:54206)
0807 18:20:35.922641 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:35.922816 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:33067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260807 18:20:35.924105 18088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:35.923719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:33067 (local address 127.0.0.1:54216)
0807 18:20:35.923882 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:35.923968 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:33067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 18:20:35.924438 18078 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.60:33067: Network error: Client connection negotiation failed: client connection to 127.17.110.60:33067: connect: Connection refused (error 111)
I20260807 18:20:35.948414 17977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.60" port: 33067 } attempt: 2
I20260807 18:20:35.949846 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:35.949289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:33067 (local address 127.0.0.1:54230)
0807 18:20:35.949562 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:35.949664 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:33067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260807 18:20:35.951126 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:35.950671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:33067 (local address 127.0.0.1:54232)
0807 18:20:35.950874 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:35.950962 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:33067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260807 18:20:35.951382 17977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.60:33067: Network error: Client connection negotiation failed: client connection to 127.17.110.60:33067: connect: Connection refused (error 111)
I20260807 18:20:36.015761 18078 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.60" port: 33067 } attempt: 3
I20260807 18:20:36.017390 18088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:36.016677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:33067 (local address 127.0.0.1:54244)
0807 18:20:36.016976 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:36.017134 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:33067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 18:20:36.019148 18088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:36.018302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:33067 (local address 127.0.0.1:54256)
0807 18:20:36.018764 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:36.018913 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:33067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239}
W20260807 18:20:36.019508 18078 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.60:33067: Network error: Client connection negotiation failed: client connection to 127.17.110.60:33067: connect: Connection refused (error 111)
I20260807 18:20:36.057654 17977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.60" port: 33067 } attempt: 3
I20260807 18:20:36.059064 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:36.058447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:33067 (local address 127.0.0.1:54268)
0807 18:20:36.058762 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:36.058868 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:33067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
I20260807 18:20:36.060582 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:36.060029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:33067 (local address 127.0.0.1:54284)
0807 18:20:36.060306 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:36.060413 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:33067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
W20260807 18:20:36.060981 17977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.60:33067: Network error: Client connection negotiation failed: client connection to 127.17.110.60:33067: connect: Connection refused (error 111)
I20260807 18:20:36.162873 18078 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.60" port: 33067 } attempt: 4
I20260807 18:20:36.164342 18088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:36.163654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:33067 (local address 127.0.0.1:54296)
0807 18:20:36.163979 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:36.164107 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:33067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260807 18:20:36.166124 18088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:36.165493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:33067 (local address 127.0.0.1:54310)
0807 18:20:36.165736 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:36.165870 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:33067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260807 18:20:36.166579 18078 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.60:33067: Network error: Client connection negotiation failed: client connection to 127.17.110.60:33067: connect: Connection refused (error 111)
W20260807 18:20:36.228811 18095 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:20:36.229223 18095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:20:36.229295 18095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:20:36.229383 18095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:20:36.237866 18095 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:20:36.238006 18095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:20:36.238070 18095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:20:36.238118 18095 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:20:36.238178 18095 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:20:36.238382 17977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.60" port: 33067 } attempt: 4
I20260807 18:20:36.239861 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:36.239240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:33067 (local address 127.0.0.1:54316)
0807 18:20:36.239550 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:36.239667 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:33067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
I20260807 18:20:36.241164 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:36.240730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:33067 (local address 127.0.0.1:54330)
0807 18:20:36.240921 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:36.241001 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:33067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 18:20:36.241495 17977 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.60:33067: Network error: Client connection negotiation failed: client connection to 127.17.110.60:33067: connect: Connection refused (error 111)
I20260807 18:20:36.250157 18095 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:32781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:37121,127.17.110.61:45869,127.17.110.60:33067
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.60.keytab
--principal=kudu/127.17.110.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.60:33067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.110.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:20:36.251793 18095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:20:36.253397 18095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:20:36.268019 18101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:36.267939 18102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:36.267984 18114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:20:36.269263 18095 server_base.cc:1034] running on GCE node
I20260807 18:20:36.270155 18095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:20:36.271911 18095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:20:36.273242 18095 hybrid_clock.cc:648] HybridClock initialized: now 1786126836273136 us; error 69 us; skew 500 ppm
I20260807 18:20:36.273643 18095 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:20:36 dist-test-slave-39l8 krb5kdc[17865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126836, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:20:36.277447 18095 init.cc:377] Logged in from keytab as kudu/127.17.110.60@KRBTEST.COM (short username kudu)
I20260807 18:20:36.279292 18095 webserver.cc:460] Webserver started at http://127.17.110.60:34357/ using document root <none> and password file <none>
I20260807 18:20:36.280057 18095 fs_manager.cc:359] Metadata directory not provided
I20260807 18:20:36.280157 18095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:20:36.280527 18095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:20:36.283990 18095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-2/data/instance:
uuid: "a4a5d252b8944ef8a4bcc119c20a6505"
format_stamp: "Formatted at 2026-08-07 18:20:36 on dist-test-slave-39l8"
server_key: "5ff1a6503392cff2c8c501ae82e94027"
server_key_iv: "157d599ad8b3ac4347cdecd35e977a6c"
server_key_version: "encryptionkey@0"
I20260807 18:20:36.284854 18095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-2/wal/instance:
uuid: "a4a5d252b8944ef8a4bcc119c20a6505"
format_stamp: "Formatted at 2026-08-07 18:20:36 on dist-test-slave-39l8"
server_key: "5ff1a6503392cff2c8c501ae82e94027"
server_key_iv: "157d599ad8b3ac4347cdecd35e977a6c"
server_key_version: "encryptionkey@0"
I20260807 18:20:36.290925 18095 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 18:20:36.295009 18123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:36.296903 18095 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:20:36.297127 18095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-2/wal
uuid: "a4a5d252b8944ef8a4bcc119c20a6505"
format_stamp: "Formatted at 2026-08-07 18:20:36 on dist-test-slave-39l8"
server_key: "5ff1a6503392cff2c8c501ae82e94027"
server_key_iv: "157d599ad8b3ac4347cdecd35e977a6c"
server_key_version: "encryptionkey@0"
I20260807 18:20:36.297403 18095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:20:36.333230 18095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:20:36.338322 18095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:20:36.338683 18095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:20:36.362975 18095 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.60:33067
I20260807 18:20:36.363006 18202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.60:33067 every 8 connection(s)
I20260807 18:20:36.364559 18095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-2/data/info.pb
I20260807 18:20:36.368534 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 18095
I20260807 18:20:36.368913 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126429568977-17848-0/minicluster-data/master-2/wal/instance
I20260807 18:20:36.369647 17848 external_mini_cluster.cc:1428] Setting key 75db8c7a19b8e5d8e2ef2b84a8c36a0d
I20260807 18:20:36.370108 18203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:20:36.382110 18203 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.110.62" port: 37121 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:20:36 dist-test-slave-39l8 krb5kdc[17865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.60@KRBTEST.COM
Aug 07 18:20:36 dist-test-slave-39l8 krb5kdc[17865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126836, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.60@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:20:36.400949 18206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:36.372065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54338 (local address 127.17.110.60:33067)
0807 18:20:36.372990 (+ 925us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:36.373004 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:36.373054 (+ 50us) server_negotiation.cc:407] Connection header received
0807 18:20:36.376117 (+ 3063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:36.376156 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:36.376737 (+ 581us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:36.377405 (+ 668us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:36.379384 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:36.381111 (+ 1727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:36.383028 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:36.384682 (+ 1654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:36.384948 (+ 266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:36.390938 (+ 5990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:36.390948 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:36.390966 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:36.391030 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:36.395012 (+ 3982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:36.396089 (+ 1077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:36.396099 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:36.396110 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:36.396347 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:36.396844 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:36.396850 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:36.396853 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:36.397585 (+ 732us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:36.397962 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:36.398456 (+ 494us) server_negotiation.cc:299] Negotiation successful
0807 18:20:36.399039 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":685,"thread_start_us":397,"threads_started":1}
I20260807 18:20:36.405985 18207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:36.385699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35322 (local address 127.17.110.62:37121)
0807 18:20:36.387891 (+ 2192us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:36.387900 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:36.387933 (+ 33us) server_negotiation.cc:407] Connection header received
0807 18:20:36.388012 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:36.388020 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:36.388120 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:36.388296 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:36.389781 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:36.390520 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:36.392310 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:36.393060 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:36.393219 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:36.397418 (+ 4199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:36.397425 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:36.397430 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:36.397487 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:36.401914 (+ 4427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:36.403224 (+ 1310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:36.403229 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:36.403231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:36.403392 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:36.403881 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:36.404009 (+ 128us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:36.404016 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:36.404448 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:36.404617 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:36.405252 (+ 635us) server_negotiation.cc:299] Negotiation successful
0807 18:20:36.405516 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1916,"thread_start_us":1748,"threads_started":1}
I20260807 18:20:36.405977 18208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:36.385702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:37121 (local address 127.0.0.1:35322)
0807 18:20:36.386857 (+ 1155us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:36.386935 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0807 18:20:36.387158 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:20:36.388469 (+ 1311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:20:36.388475 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:20:36.389201 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:20:36.389506 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:36.389519 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:36.390791 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:36.390796 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:36.392083 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:36.392100 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:36.393320 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:36.393336 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:36.393542 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:36.394260 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:20:36.394280 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:20:36.397111 (+ 2831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:20:36.402154 (+ 5043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:36.402170 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:36.402179 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:36.403025 (+ 846us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:36.403531 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:36.403536 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:36.403540 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:36.403726 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:36.404629 (+ 903us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:20:36.404638 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:20:36.404874 (+ 236us) client_negotiation.cc:769] Sending connection context
0807 18:20:36.405091 (+ 217us) client_negotiation.cc:240] Negotiation successful
0807 18:20:36.405482 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":733,"thread_start_us":523,"threads_started":1}
I20260807 18:20:36.408077 18203 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.110.61" port: 45869 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:20:36 dist-test-slave-39l8 krb5kdc[17865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
Aug 07 18:20:36 dist-test-slave-39l8 krb5kdc[17865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126836, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.60@KRBTEST.COM for kudu/127.17.110.61@KRBTEST.COM
I20260807 18:20:36.425310 18209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:36.405350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35328 (local address 127.17.110.62:37121)
0807 18:20:36.406411 (+ 1061us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:36.406418 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:36.406447 (+ 29us) server_negotiation.cc:407] Connection header received
0807 18:20:36.406503 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:36.406508 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:36.406582 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:36.406753 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:36.408229 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:36.409334 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:36.410893 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:36.411376 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:36.411533 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:36.417254 (+ 5721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:36.417262 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:36.417266 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:36.417312 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:36.421146 (+ 3834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:36.422170 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:36.422177 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:36.422181 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:36.422387 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:36.422955 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:36.422959 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:36.422962 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:36.423406 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:36.423682 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:36.424561 (+ 879us) server_negotiation.cc:299] Negotiation successful
0807 18:20:36.424882 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":896,"thread_start_us":652,"threads_started":1}
I20260807 18:20:36.426179 18208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:36.408897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:45869 (local address 127.0.0.1:60416)
0807 18:20:36.409085 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:36.409106 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:20:36.409222 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:20:36.410078 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:20:36.410083 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:20:36.410567 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:20:36.410805 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:36.410817 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:36.411934 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:36.411940 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:36.413100 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:36.413117 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:36.413927 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:36.413934 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:36.414137 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:36.414832 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:20:36.414852 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:20:36.417513 (+ 2661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:20:36.422328 (+ 4815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:36.422334 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:36.422337 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:36.423037 (+ 700us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:36.423847 (+ 810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:36.423853 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:36.423857 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:36.424084 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:36.425115 (+ 1031us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:20:36.425120 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:20:36.425287 (+ 167us) client_negotiation.cc:769] Sending connection context
0807 18:20:36.425473 (+ 186us) client_negotiation.cc:240] Negotiation successful
0807 18:20:36.425808 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 18:20:36.426393 18210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:36.409209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60416 (local address 127.17.110.61:45869)
0807 18:20:36.409778 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:36.409782 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:36.409799 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:20:36.409849 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:36.409853 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:36.409909 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:36.410050 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:36.411021 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:36.411770 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:36.413298 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:36.413742 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:36.413900 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:36.417826 (+ 3926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:36.417835 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:36.417842 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:36.417895 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:36.422019 (+ 4124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:36.423193 (+ 1174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:36.423199 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:36.423202 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:36.423401 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:36.424460 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:36.424466 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:36.424469 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:36.424896 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:36.425079 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:36.425660 (+ 581us) server_negotiation.cc:299] Negotiation successful
0807 18:20:36.425935 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":355,"threads_started":1}
I20260807 18:20:36.427625 18203 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.110.60" port: 33067 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:20:36 dist-test-slave-39l8 krb5kdc[17865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126836, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.60@KRBTEST.COM for kudu/127.17.110.60@KRBTEST.COM
Aug 07 18:20:36 dist-test-slave-39l8 krb5kdc[17865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.61@KRBTEST.COM
I20260807 18:20:36.445587 18208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:36.428529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:33067 (local address 127.0.0.1:54344)
0807 18:20:36.428747 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:36.428896 (+ 149us) client_negotiation.cc:174] Beginning negotiation
0807 18:20:36.429047 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:20:36.429608 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:20:36.429613 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:20:36.430110 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:20:36.430260 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:36.430269 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:36.431656 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:36.431662 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:36.433166 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:36.433185 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:36.434270 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:36.434276 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:36.434510 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:36.435359 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:20:36.435386 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:20:36.438337 (+ 2951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:20:36.442380 (+ 4043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:36.442389 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:36.442394 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:36.443029 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:36.443549 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:36.443557 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:36.443564 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:36.443751 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:36.444350 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:20:36.444355 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:20:36.444546 (+ 191us) client_negotiation.cc:769] Sending connection context
0807 18:20:36.444747 (+ 201us) client_negotiation.cc:240] Negotiation successful
0807 18:20:36.445230 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"mutex_wait_us":38}
I20260807 18:20:36.446105 18206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:36.428731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54344 (local address 127.17.110.60:33067)
0807 18:20:36.428860 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:36.428866 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:36.429013 (+ 147us) server_negotiation.cc:407] Connection header received
0807 18:20:36.429170 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:36.429174 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:36.429440 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:36.429540 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:36.430511 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:36.431516 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:36.433336 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:36.434072 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:36.434199 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:36.438762 (+ 4563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:36.438784 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:36.438791 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:36.438842 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:36.442169 (+ 3327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:36.443246 (+ 1077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:36.443251 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:36.443253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:36.443391 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:36.443904 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:36.443909 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:36.443911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:36.444236 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:36.444378 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:36.444849 (+ 471us) server_negotiation.cc:299] Negotiation successful
0807 18:20:36.445482 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":279,"server-negotiator.queue_time_us":36}
I20260807 18:20:36.448697 18210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:36.427516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60422 (local address 127.17.110.61:45869)
0807 18:20:36.427727 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:36.427735 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:36.427766 (+ 31us) server_negotiation.cc:407] Connection header received
0807 18:20:36.427916 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:36.427922 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:36.428002 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:36.428124 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:36.429964 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:36.431231 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:36.433240 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:36.433989 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:36.434184 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:36.440176 (+ 5992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:36.440183 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:36.440187 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:36.440230 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:36.444098 (+ 3868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:36.445189 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:36.445194 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:36.445197 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:36.445430 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:36.446182 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:36.446189 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:36.446192 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:36.446586 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:36.446973 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:36.447661 (+ 688us) server_negotiation.cc:299] Negotiation successful
0807 18:20:36.448104 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 18:20:36.452986 18203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4a5d252b8944ef8a4bcc119c20a6505: Bootstrap starting.
I20260807 18:20:36.455030 17848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:20:36.456722 18203 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4a5d252b8944ef8a4bcc119c20a6505: Neither blocks nor log segments found. Creating new log.
I20260807 18:20:36.458125 18203 log.cc:824] T 00000000000000000000000000000000 P a4a5d252b8944ef8a4bcc119c20a6505: Log is configured to *not* fsync() on all Append() calls
I20260807 18:20:36.458065 17939 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b4fc9dfe61b34502a69682b4251e1a43"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35328
I20260807 18:20:36.461845 18203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4a5d252b8944ef8a4bcc119c20a6505: No bootstrap required, opened a new log
I20260807 18:20:36.465878 18078 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.60" port: 33067 } attempt: 5
I20260807 18:20:36.472530 18203 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4a5d252b8944ef8a4bcc119c20a6505 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc9dfe61b34502a69682b4251e1a43" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 37121 } } peers { permanent_uuid: "69dbc49add32419e8509237bda6639d2" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 45869 } } peers { permanent_uuid: "a4a5d252b8944ef8a4bcc119c20a6505" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 33067 } }
I20260807 18:20:36.473006 18203 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4a5d252b8944ef8a4bcc119c20a6505 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4a5d252b8944ef8a4bcc119c20a6505, State: Initialized, Role: FOLLOWER
I20260807 18:20:36.473628 18203 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4a5d252b8944ef8a4bcc119c20a6505 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc9dfe61b34502a69682b4251e1a43" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 37121 } } peers { permanent_uuid: "69dbc49add32419e8509237bda6639d2" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 45869 } } peers { permanent_uuid: "a4a5d252b8944ef8a4bcc119c20a6505" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 33067 } }
I20260807 18:20:36.474771 18212 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4a5d252b8944ef8a4bcc119c20a6505 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc9dfe61b34502a69682b4251e1a43" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 37121 } } peers { permanent_uuid: "69dbc49add32419e8509237bda6639d2" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 45869 } } peers { permanent_uuid: "a4a5d252b8944ef8a4bcc119c20a6505" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 33067 } } }
I20260807 18:20:36.475183 18212 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4a5d252b8944ef8a4bcc119c20a6505 [sys.catalog]: This master's current role is: FOLLOWER
Aug 07 18:20:36 dist-test-slave-39l8 krb5kdc[17865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126835, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.61@KRBTEST.COM for kudu/127.17.110.60@KRBTEST.COM
I20260807 18:20:36.476202 18203 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4a5d252b8944ef8a4bcc119c20a6505 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:20:36.485047 18088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:36.466742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:33067 (local address 127.0.0.1:54358)
0807 18:20:36.466967 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:36.466989 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:20:36.467118 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:20:36.467606 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:20:36.467611 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:20:36.468142 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:20:36.468364 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:36.468378 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:36.469966 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:36.469972 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:36.471466 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:36.471489 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:36.472659 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:36.472668 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:36.473160 (+ 492us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:36.473933 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:20:36.473968 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:20:36.476907 (+ 2939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:20:36.481617 (+ 4710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:36.481624 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:36.481629 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:36.482336 (+ 707us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:36.482926 (+ 590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:36.482933 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:36.482938 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:36.483139 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:36.483825 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:20:36.483830 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:20:36.484020 (+ 190us) client_negotiation.cc:769] Sending connection context
0807 18:20:36.484197 (+ 177us) client_negotiation.cc:240] Negotiation successful
0807 18:20:36.484603 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260807 18:20:36.485055 18206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:36.466844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54358 (local address 127.17.110.60:33067)
0807 18:20:36.467055 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:36.467061 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:36.467082 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:20:36.467309 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:36.467315 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:36.467412 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:36.467514 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:36.468544 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:36.469793 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:36.471668 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:36.472429 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:36.472596 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:36.477149 (+ 4553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:36.477165 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:36.477170 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:36.477219 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:36.481404 (+ 4185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:36.482525 (+ 1121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:36.482530 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:36.482533 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:36.482766 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:36.483294 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:36.483300 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:36.483304 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:36.483668 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:36.483895 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:36.484340 (+ 445us) server_negotiation.cc:299] Negotiation successful
0807 18:20:36.484599 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:20:36.492082 18078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69dbc49add32419e8509237bda6639d2: Bootstrap starting.
I20260807 18:20:36.497623 18078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69dbc49add32419e8509237bda6639d2: Neither blocks nor log segments found. Creating new log.
I20260807 18:20:36.499946 18078 log.cc:824] T 00000000000000000000000000000000 P 69dbc49add32419e8509237bda6639d2: Log is configured to *not* fsync() on all Append() calls
W20260807 18:20:36.505949 18229 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a4a5d252b8944ef8a4bcc119c20a6505: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:20:36.506166 18229 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:20:36.492111 18095 master_runner.cc:186] Error getting master registration for 127.17.110.62:37121: OK, instance_id { permanent_uuid: "b4fc9dfe61b34502a69682b4251e1a43" instance_seqno: 1786126835070336 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:20:36.509184 18095 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 18:20:36.511010 18078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69dbc49add32419e8509237bda6639d2: No bootstrap required, opened a new log
I20260807 18:20:36.518815 17977 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.60" port: 33067 } attempt: 5
I20260807 18:20:36.523224 18078 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69dbc49add32419e8509237bda6639d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc9dfe61b34502a69682b4251e1a43" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 37121 } } peers { permanent_uuid: "69dbc49add32419e8509237bda6639d2" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 45869 } } peers { permanent_uuid: "a4a5d252b8944ef8a4bcc119c20a6505" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 33067 } }
I20260807 18:20:36.523725 18078 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69dbc49add32419e8509237bda6639d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69dbc49add32419e8509237bda6639d2, State: Initialized, Role: FOLLOWER
I20260807 18:20:36.524365 18078 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69dbc49add32419e8509237bda6639d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc9dfe61b34502a69682b4251e1a43" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 37121 } } peers { permanent_uuid: "69dbc49add32419e8509237bda6639d2" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 45869 } } peers { permanent_uuid: "a4a5d252b8944ef8a4bcc119c20a6505" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 33067 } }
I20260807 18:20:36.525995 18235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69dbc49add32419e8509237bda6639d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc9dfe61b34502a69682b4251e1a43" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 37121 } } peers { permanent_uuid: "69dbc49add32419e8509237bda6639d2" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 45869 } } peers { permanent_uuid: "a4a5d252b8944ef8a4bcc119c20a6505" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 33067 } } }
I20260807 18:20:36.527104 18235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69dbc49add32419e8509237bda6639d2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:20:36.527539 18078 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69dbc49add32419e8509237bda6639d2 [sys.catalog]: configured and running, proceeding with master startup.
Aug 07 18:20:36 dist-test-slave-39l8 krb5kdc[17865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126834, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for kudu/127.17.110.60@KRBTEST.COM
I20260807 18:20:36.539825 18206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:36.520122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54372 (local address 127.17.110.60:33067)
0807 18:20:36.520310 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:36.520318 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:36.520338 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:20:36.520402 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:36.520407 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:36.520483 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:36.520592 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:36.521554 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:36.522772 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:36.525080 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:36.526081 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:36.526250 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:36.531623 (+ 5373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:36.531632 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:36.531641 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:36.531698 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:36.536571 (+ 4873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:36.537442 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:36.537450 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:36.537454 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:36.537670 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:36.538133 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:36.538140 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:36.538145 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:36.538524 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:36.538688 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:36.539178 (+ 490us) server_negotiation.cc:299] Negotiation successful
0807 18:20:36.539431 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:20:36.539848 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:36.519687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:33067 (local address 127.0.0.1:54372)
0807 18:20:36.519892 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:36.519916 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:20:36.520055 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:20:36.520633 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:20:36.520638 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:20:36.521109 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:20:36.521370 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:36.521381 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:36.522996 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:36.523002 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:36.524776 (+ 1774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:36.524816 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:36.526580 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:36.526588 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:36.526882 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:36.527739 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:20:36.527781 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:20:36.531273 (+ 3492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:20:36.536775 (+ 5502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:36.536782 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:36.536786 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:36.537273 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:36.537797 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:36.537803 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:36.537807 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:36.537998 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:36.538650 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:20:36.538655 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:20:36.538866 (+ 211us) client_negotiation.cc:769] Sending connection context
0807 18:20:36.539035 (+ 169us) client_negotiation.cc:240] Negotiation successful
0807 18:20:36.539377 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 18:20:36.545419 17977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4fc9dfe61b34502a69682b4251e1a43: Bootstrap starting.
I20260807 18:20:36.556492 17977 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4fc9dfe61b34502a69682b4251e1a43: Neither blocks nor log segments found. Creating new log.
I20260807 18:20:36.558386 17977 log.cc:824] T 00000000000000000000000000000000 P b4fc9dfe61b34502a69682b4251e1a43: Log is configured to *not* fsync() on all Append() calls
W20260807 18:20:36.566757 18249 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 69dbc49add32419e8509237bda6639d2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:20:36.567363 18249 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:20:36.569301 17977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4fc9dfe61b34502a69682b4251e1a43: No bootstrap required, opened a new log
I20260807 18:20:36.552268 17984 master_runner.cc:186] Error getting master registration for 127.17.110.62:37121: OK, instance_id { permanent_uuid: "b4fc9dfe61b34502a69682b4251e1a43" instance_seqno: 1786126835070336 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:20:36.578516 17984 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 18:20:36.578295 17977 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4fc9dfe61b34502a69682b4251e1a43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc9dfe61b34502a69682b4251e1a43" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 37121 } } peers { permanent_uuid: "69dbc49add32419e8509237bda6639d2" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 45869 } } peers { permanent_uuid: "a4a5d252b8944ef8a4bcc119c20a6505" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 33067 } }
I20260807 18:20:36.578913 17977 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4fc9dfe61b34502a69682b4251e1a43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4fc9dfe61b34502a69682b4251e1a43, State: Initialized, Role: FOLLOWER
I20260807 18:20:36.579573 17977 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4fc9dfe61b34502a69682b4251e1a43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc9dfe61b34502a69682b4251e1a43" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 37121 } } peers { permanent_uuid: "69dbc49add32419e8509237bda6639d2" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 45869 } } peers { permanent_uuid: "a4a5d252b8944ef8a4bcc119c20a6505" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 33067 } }
I20260807 18:20:36.586813 18252 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4fc9dfe61b34502a69682b4251e1a43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc9dfe61b34502a69682b4251e1a43" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 37121 } } peers { permanent_uuid: "69dbc49add32419e8509237bda6639d2" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 45869 } } peers { permanent_uuid: "a4a5d252b8944ef8a4bcc119c20a6505" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 33067 } } }
I20260807 18:20:36.587448 18252 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4fc9dfe61b34502a69682b4251e1a43 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:20:36.589007 17977 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4fc9dfe61b34502a69682b4251e1a43 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:20:36.589063 17939 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4fc9dfe61b34502a69682b4251e1a43 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 18:20:36.589291 17939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4fc9dfe61b34502a69682b4251e1a43 [term 0 FOLLOWER]: Advancing to term 1
W20260807 18:20:36.591333 18135 tablet.cc:2369] T 00000000000000000000000000000000 P a4a5d252b8944ef8a4bcc119c20a6505: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:20:36.594116 17939 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4fc9dfe61b34502a69682b4251e1a43 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc9dfe61b34502a69682b4251e1a43" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 37121 } } peers { permanent_uuid: "69dbc49add32419e8509237bda6639d2" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 45869 } } peers { permanent_uuid: "a4a5d252b8944ef8a4bcc119c20a6505" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 33067 } }
I20260807 18:20:36.597760 18044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b4fc9dfe61b34502a69682b4251e1a43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "69dbc49add32419e8509237bda6639d2"
I20260807 18:20:36.598454 18044 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69dbc49add32419e8509237bda6639d2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:20:36.604121 18044 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 69dbc49add32419e8509237bda6639d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4fc9dfe61b34502a69682b4251e1a43 in term 1.
I20260807 18:20:36.605366 17905 leader_election.cc:302] T 00000000000000000000000000000000 P b4fc9dfe61b34502a69682b4251e1a43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69dbc49add32419e8509237bda6639d2, b4fc9dfe61b34502a69682b4251e1a43; no voters:
I20260807 18:20:36.607859 18163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b4fc9dfe61b34502a69682b4251e1a43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a4a5d252b8944ef8a4bcc119c20a6505"
I20260807 18:20:36.608669 18163 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4a5d252b8944ef8a4bcc119c20a6505 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:20:36.614550 18163 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a4a5d252b8944ef8a4bcc119c20a6505 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4fc9dfe61b34502a69682b4251e1a43 in term 1.
I20260807 18:20:36.616046 18252 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4fc9dfe61b34502a69682b4251e1a43 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:20:36.616379 17939 leader_election.cc:288] T 00000000000000000000000000000000 P b4fc9dfe61b34502a69682b4251e1a43 [CANDIDATE]: Term 1 election: Requested vote from peers 69dbc49add32419e8509237bda6639d2 (127.17.110.61:45869), a4a5d252b8944ef8a4bcc119c20a6505 (127.17.110.60:33067)
I20260807 18:20:36.635500 18252 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4fc9dfe61b34502a69682b4251e1a43 [term 1 LEADER]: Becoming Leader. State: Replica: b4fc9dfe61b34502a69682b4251e1a43, State: Running, Role: LEADER
I20260807 18:20:36.637204 18252 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4fc9dfe61b34502a69682b4251e1a43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc9dfe61b34502a69682b4251e1a43" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 37121 } } peers { permanent_uuid: "69dbc49add32419e8509237bda6639d2" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 45869 } } peers { permanent_uuid: "a4a5d252b8944ef8a4bcc119c20a6505" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 33067 } }
I20260807 18:20:36.654232 18262 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4fc9dfe61b34502a69682b4251e1a43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4fc9dfe61b34502a69682b4251e1a43. Latest consensus state: current_term: 1 leader_uuid: "b4fc9dfe61b34502a69682b4251e1a43" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc9dfe61b34502a69682b4251e1a43" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 37121 } } peers { permanent_uuid: "69dbc49add32419e8509237bda6639d2" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 45869 } } peers { permanent_uuid: "a4a5d252b8944ef8a4bcc119c20a6505" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 33067 } } }
I20260807 18:20:36.654992 18262 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4fc9dfe61b34502a69682b4251e1a43 [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:36.665506 18275 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:20:36.695186 18275 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260807 18:20:36.701825 18011 tablet.cc:2369] T 00000000000000000000000000000000 P 69dbc49add32419e8509237bda6639d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:20:36.720139 18206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:36.673170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54378 (local address 127.17.110.60:33067)
0807 18:20:36.673383 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:36.673389 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:36.678812 (+ 5423us) server_negotiation.cc:407] Connection header received
0807 18:20:36.680551 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:36.680561 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:36.680711 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:36.680806 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:36.697852 (+ 17046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:36.699557 (+ 1705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:36.701955 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:36.703420 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:36.703553 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:36.709625 (+ 6072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:36.709633 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:36.709639 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:36.709695 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:36.714252 (+ 4557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:36.715555 (+ 1303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:36.715563 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:36.715567 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:36.715753 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:36.717829 (+ 2076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:36.717837 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:36.717841 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:36.718340 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:36.718644 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:36.719405 (+ 761us) server_negotiation.cc:299] Negotiation successful
0807 18:20:36.719719 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 18:20:36.728999 18210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:36.682062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60436 (local address 127.17.110.61:45869)
0807 18:20:36.682311 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:36.682318 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:36.690965 (+ 8647us) server_negotiation.cc:407] Connection header received
0807 18:20:36.691292 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:36.691299 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:36.691399 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:36.691479 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:36.694136 (+ 2657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:36.695559 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:36.702549 (+ 6990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:36.716597 (+ 14048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:36.716773 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:36.720662 (+ 3889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:36.720672 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:36.720676 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:36.720741 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:36.724632 (+ 3891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:36.725707 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:36.725732 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:36.725736 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:36.725953 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:36.726442 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:36.726447 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:36.726450 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:36.726921 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:36.727215 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:36.728299 (+ 1084us) server_negotiation.cc:299] Negotiation successful
0807 18:20:36.728554 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:20:36.733771 18163 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a4a5d252b8944ef8a4bcc119c20a6505 [term 1 FOLLOWER]: Refusing update from remote peer b4fc9dfe61b34502a69682b4251e1a43: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:20:36.735424 18262 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b4fc9dfe61b34502a69682b4251e1a43 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4a5d252b8944ef8a4bcc119c20a6505" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 33067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:20:36.738111 18044 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 69dbc49add32419e8509237bda6639d2 [term 1 FOLLOWER]: Refusing update from remote peer b4fc9dfe61b34502a69682b4251e1a43: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:20:36.740186 18262 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b4fc9dfe61b34502a69682b4251e1a43 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69dbc49add32419e8509237bda6639d2" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 45869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:20:36.758358 18209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:36.672398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35334 (local address 127.17.110.62:37121)
0807 18:20:36.673005 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:36.673013 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:36.684731 (+ 11718us) server_negotiation.cc:407] Connection header received
0807 18:20:36.687387 (+ 2656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:36.687399 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:36.689025 (+ 1626us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:36.689173 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:36.702975 (+ 13802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:36.704594 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:36.717945 (+ 13351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:36.719001 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:36.719174 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:36.725777 (+ 6603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:36.725784 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:36.725789 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:36.725836 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:36.748664 (+ 22828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:36.753731 (+ 5067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:36.753739 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:36.753744 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:36.753964 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:36.755975 (+ 2011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:36.755982 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:36.755986 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:36.756546 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:36.756860 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:36.757651 (+ 791us) server_negotiation.cc:299] Negotiation successful
0807 18:20:36.757969 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401}
I20260807 18:20:36.766191 18292 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 18:20:36.777647 18235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69dbc49add32419e8509237bda6639d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4fc9dfe61b34502a69682b4251e1a43. Latest consensus state: current_term: 1 leader_uuid: "b4fc9dfe61b34502a69682b4251e1a43" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc9dfe61b34502a69682b4251e1a43" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 37121 } } peers { permanent_uuid: "69dbc49add32419e8509237bda6639d2" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 45869 } } peers { permanent_uuid: "a4a5d252b8944ef8a4bcc119c20a6505" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 33067 } } }
I20260807 18:20:36.777958 18235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69dbc49add32419e8509237bda6639d2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:20:36.785686 18212 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4a5d252b8944ef8a4bcc119c20a6505 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4fc9dfe61b34502a69682b4251e1a43. Latest consensus state: current_term: 1 leader_uuid: "b4fc9dfe61b34502a69682b4251e1a43" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc9dfe61b34502a69682b4251e1a43" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 37121 } } peers { permanent_uuid: "69dbc49add32419e8509237bda6639d2" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 45869 } } peers { permanent_uuid: "a4a5d252b8944ef8a4bcc119c20a6505" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 33067 } } }
I20260807 18:20:36.786046 18212 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4a5d252b8944ef8a4bcc119c20a6505 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:20:36.786725 18285 mvcc.cc:205] Tried to move back new op lower bound from 7315975523232735232 to 7315975522887401472. Current Snapshot: MvccSnapshot[applied={T|T < 7315975523232735232}]
I20260807 18:20:36.791905 18262 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4fc9dfe61b34502a69682b4251e1a43 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b4fc9dfe61b34502a69682b4251e1a43" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc9dfe61b34502a69682b4251e1a43" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 37121 } } peers { permanent_uuid: "69dbc49add32419e8509237bda6639d2" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 45869 } } peers { permanent_uuid: "a4a5d252b8944ef8a4bcc119c20a6505" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 33067 } } }
I20260807 18:20:36.792276 18262 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4fc9dfe61b34502a69682b4251e1a43 [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:36.792563 18262 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4fc9dfe61b34502a69682b4251e1a43 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b4fc9dfe61b34502a69682b4251e1a43" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc9dfe61b34502a69682b4251e1a43" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 37121 } } peers { permanent_uuid: "69dbc49add32419e8509237bda6639d2" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 45869 } } peers { permanent_uuid: "a4a5d252b8944ef8a4bcc119c20a6505" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 33067 } } }
I20260807 18:20:36.792801 18262 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4fc9dfe61b34502a69682b4251e1a43 [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:36.795403 18235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69dbc49add32419e8509237bda6639d2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b4fc9dfe61b34502a69682b4251e1a43" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc9dfe61b34502a69682b4251e1a43" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 37121 } } peers { permanent_uuid: "69dbc49add32419e8509237bda6639d2" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 45869 } } peers { permanent_uuid: "a4a5d252b8944ef8a4bcc119c20a6505" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 33067 } } }
I20260807 18:20:36.795678 18235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69dbc49add32419e8509237bda6639d2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:20:36.800750 18275 catalog_manager.cc:1347] Generated new cluster ID: 16abb45524d248ce93400acb9aefdd57
I20260807 18:20:36.800916 18275 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:20:36.807941 18290 mvcc.cc:205] Tried to move back new op lower bound from 7315975523232735232 to 7315975522887401472. Current Snapshot: MvccSnapshot[applied={T|T < 7315975523232735232}]
I20260807 18:20:36.808885 18212 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4a5d252b8944ef8a4bcc119c20a6505 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b4fc9dfe61b34502a69682b4251e1a43" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc9dfe61b34502a69682b4251e1a43" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 37121 } } peers { permanent_uuid: "69dbc49add32419e8509237bda6639d2" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 45869 } } peers { permanent_uuid: "a4a5d252b8944ef8a4bcc119c20a6505" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 33067 } } }
I20260807 18:20:36.809191 18212 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4a5d252b8944ef8a4bcc119c20a6505 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:20:36.819890 18275 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:20:36.821820 18275 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:20:36.844466 18275 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4fc9dfe61b34502a69682b4251e1a43: Generated new TSK 0
I20260807 18:20:36.846064 18275 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 18:20:37.000470 18206 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:20:36.988048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54392 (local address 127.17.110.60:33067)
0807 18:20:36.988298 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:36.988307 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:36.998052 (+ 9745us) server_negotiation.cc:407] Connection header received
0807 18:20:36.998626 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:36.998633 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:36.998767 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:36.998857 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:37.000168 (+ 1311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54392: BlockingRecv error: recv got EOF from 127.0.0.1:54392 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260807 18:20:37.003724 18209 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:20:36.992150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35338 (local address 127.17.110.62:37121)
0807 18:20:36.992389 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:36.992397 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:37.001613 (+ 9216us) server_negotiation.cc:407] Connection header received
0807 18:20:37.001945 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:37.001957 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:37.002051 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:37.002139 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:37.003387 (+ 1248us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35338: BlockingRecv error: recv got EOF from 127.0.0.1:35338 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260807 18:20:37.008030 18210 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:20:37.005154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60442 (local address 127.17.110.61:45869)
0807 18:20:37.005396 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:37.005402 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:37.006164 (+ 762us) server_negotiation.cc:407] Connection header received
0807 18:20:37.006458 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:37.006464 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:37.006558 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:37.006636 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:37.007752 (+ 1116us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60442: BlockingRecv error: recv got EOF from 127.0.0.1:60442 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260807 18:20:37.055692 18210 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:20:37.045066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60444 (local address 127.17.110.61:45869)
0807 18:20:37.045339 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:37.045346 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:37.052493 (+ 7147us) server_negotiation.cc:407] Connection header received
0807 18:20:37.053659 (+ 1166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:37.053666 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:37.053877 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:37.053968 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:37.055387 (+ 1419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60444: BlockingRecv error: recv got EOF from 127.0.0.1:60444 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260807 18:20:37.101049 18210 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:20:37.091008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60458 (local address 127.17.110.61:45869)
0807 18:20:37.091259 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:37.091268 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:37.098557 (+ 7289us) server_negotiation.cc:407] Connection header received
0807 18:20:37.099263 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:37.099270 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:37.099390 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:37.099475 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:37.100800 (+ 1325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60458: BlockingRecv error: recv got EOF from 127.0.0.1:60458 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
W20260807 18:20:37.151294 18210 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:20:37.141356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60468 (local address 127.17.110.61:45869)
0807 18:20:37.141583 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:37.141590 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:37.148781 (+ 7191us) server_negotiation.cc:407] Connection header received
0807 18:20:37.149450 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:37.149457 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:37.149566 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:37.149655 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:37.150952 (+ 1297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60468: BlockingRecv error: recv got EOF from 127.0.0.1:60468 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260807 18:20:37.203559 18210 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:20:37.193283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60472 (local address 127.17.110.61:45869)
0807 18:20:37.193549 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:37.193556 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:37.200903 (+ 7347us) server_negotiation.cc:407] Connection header received
0807 18:20:37.201623 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:37.201630 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:37.201761 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:37.201852 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:37.203297 (+ 1445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60472: BlockingRecv error: recv got EOF from 127.0.0.1:60472 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260807 18:20:37.258148 18210 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:20:37.247515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60482 (local address 127.17.110.61:45869)
0807 18:20:37.247815 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:37.247823 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:37.255716 (+ 7893us) server_negotiation.cc:407] Connection header received
0807 18:20:37.256315 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:37.256321 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:37.256434 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:37.256517 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:37.257825 (+ 1308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60482: BlockingRecv error: recv got EOF from 127.0.0.1:60482 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
W20260807 18:20:37.286769 18210 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:20:37.283313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60492 (local address 127.17.110.61:45869)
0807 18:20:37.283523 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:37.283529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:37.284686 (+ 1157us) server_negotiation.cc:407] Connection header received
0807 18:20:37.285136 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:37.285146 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:37.285274 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:37.285399 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:37.286451 (+ 1052us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60492: BlockingRecv error: recv got EOF from 127.0.0.1:60492 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260807 18:20:37.328015 18210 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:20:37.325480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60502 (local address 127.17.110.61:45869)
0807 18:20:37.325749 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:37.325755 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:37.326167 (+ 412us) server_negotiation.cc:407] Connection header received
0807 18:20:37.326409 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:37.326413 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:37.326496 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:37.326611 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:37.327740 (+ 1129us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60502: BlockingRecv error: recv got EOF from 127.0.0.1:60502 (error 108)
Metrics: {"server-negotiator.queue_time_us":123}
W20260807 18:20:37.399495 18210 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:20:37.396972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60508 (local address 127.17.110.61:45869)
0807 18:20:37.397233 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:37.397240 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:37.397647 (+ 407us) server_negotiation.cc:407] Connection header received
0807 18:20:37.397831 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:37.397835 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:37.397929 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:37.398037 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:37.399237 (+ 1200us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60508: BlockingRecv error: recv got EOF from 127.0.0.1:60508 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
I20260807 18:20:37.508440 18229 catalog_manager.cc:1259] Loaded cluster ID: 16abb45524d248ce93400acb9aefdd57
I20260807 18:20:37.508566 18229 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a4a5d252b8944ef8a4bcc119c20a6505: loading cluster ID for follower catalog manager: success
I20260807 18:20:37.511911 18229 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a4a5d252b8944ef8a4bcc119c20a6505: acquiring CA information for follower catalog manager: success
I20260807 18:20:37.514110 18229 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a4a5d252b8944ef8a4bcc119c20a6505: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 18:20:37.534209 18210 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:20:37.531756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60524 (local address 127.17.110.61:45869)
0807 18:20:37.532094 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:37.532101 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:37.532535 (+ 434us) server_negotiation.cc:407] Connection header received
0807 18:20:37.532760 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:37.532764 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:37.532865 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:37.532972 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:37.533930 (+ 958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60524: BlockingRecv error: recv got EOF from 127.0.0.1:60524 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
I20260807 18:20:37.569980 18249 catalog_manager.cc:1259] Loaded cluster ID: 16abb45524d248ce93400acb9aefdd57
I20260807 18:20:37.570109 18249 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 69dbc49add32419e8509237bda6639d2: loading cluster ID for follower catalog manager: success
I20260807 18:20:37.573385 18249 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 69dbc49add32419e8509237bda6639d2: acquiring CA information for follower catalog manager: success
I20260807 18:20:37.575841 18249 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 69dbc49add32419e8509237bda6639d2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:20:37.804121 18210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:37.797039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60540 (local address 127.17.110.61:45869)
0807 18:20:37.797352 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:37.797365 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:37.797680 (+ 315us) server_negotiation.cc:407] Connection header received
0807 18:20:37.797853 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:37.797857 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:37.798039 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:37.798205 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:20:37.798849 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:37.799710 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:37.801365 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:37.801865 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:37.801994 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:37.802834 (+ 840us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:20:37.803414 (+ 580us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:20:37.803537 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:37.803755 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 18:20:37.803849 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140}
I20260807 18:20:37.819679 18389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:37.812146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54408 (local address 127.17.110.60:33067)
0807 18:20:37.812946 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:37.812950 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:37.812970 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:20:37.813076 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:37.813080 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:37.813255 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:37.813420 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:20:37.813881 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:37.814726 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:37.816545 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:37.817012 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:37.817206 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:37.817927 (+ 721us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:20:37.818578 (+ 651us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:20:37.818859 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:37.819048 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 18:20:37.819149 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":563,"threads_started":1}
I20260807 18:20:37.845827 18389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:37.832112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54422 (local address 127.17.110.60:33067)
0807 18:20:37.832365 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:37.832374 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:37.834512 (+ 2138us) server_negotiation.cc:407] Connection header received
0807 18:20:37.834849 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:37.834859 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:37.835024 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:37.835185 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:20:37.836191 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:37.837911 (+ 1720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:37.842068 (+ 4157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:37.842890 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:37.843088 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:37.844794 (+ 1706us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:20:37.844956 (+ 162us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:20:37.845115 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:37.845352 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 18:20:37.845461 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 18:20:37.847784 18210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:37.832749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60546 (local address 127.17.110.61:45869)
0807 18:20:37.832997 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:37.833005 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:37.839016 (+ 6011us) server_negotiation.cc:407] Connection header received
0807 18:20:37.839345 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:37.839352 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:37.839476 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:37.839580 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:20:37.840547 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:37.842884 (+ 2337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:37.845181 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:37.846066 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:37.846200 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:37.846959 (+ 759us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:20:37.847101 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:20:37.847211 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:37.847459 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 18:20:37.847560 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 18:20:37.850310 18396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:37.831728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35348 (local address 127.17.110.62:37121)
0807 18:20:37.841041 (+ 9313us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:37.841059 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:37.841083 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:20:37.842168 (+ 1085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:37.842176 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:37.842443 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:37.842775 (+ 332us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:20:37.843642 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:37.845022 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:37.847792 (+ 2770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:37.848443 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:37.848598 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:37.849148 (+ 550us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:20:37.849630 (+ 482us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:20:37.849766 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:37.849924 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 18:20:37.850018 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9139,"thread_start_us":8997,"threads_started":1}
I20260807 18:20:37.863772 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 17880
I20260807 18:20:37.865391 17971 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:20:37.985980 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 17880
W20260807 18:20:38.019482 18006 connection.cc:535] server connection from 127.0.0.1:60412 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 18:20:38.020576 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 17984
I20260807 18:20:38.021858 18071 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:20:38.144141 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 17984
I20260807 18:20:38.178884 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 18095
I20260807 18:20:38.180289 18196 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:20:38.297752 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 18095
2026-08-07T18:20:38Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5562 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:20:38 dist-test-slave-39l8 krb5kdc[18407](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:20:38 dist-test-slave-39l8 krb5kdc[18407](info): set up 2 sockets
Aug 07 18:20:38 dist-test-slave-39l8 krb5kdc[18407](info): commencing operation
krb5kdc: starting...
W20260807 18:20:39.728137 17848 mini_kdc.cc:121] Time spent starting KDC: real 1.369s user 0.008s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/krb5kdc/test-user.keytab.
Aug 07 18:20:39 dist-test-slave-39l8 krb5kdc[18407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:20:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:20:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.110.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
Entry for principal kudu/127.17.110.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab.
I20260807 18:20:40.133890 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:40565
--webserver_interface=127.17.110.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:35297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:40565,127.17.110.61:35929,127.17.110.60:34417
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:20:40.547533 18424 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:20:40.548012 18424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:20:40.548312 18424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:20:40.548498 18424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:20:40.557310 18424 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:20:40.557471 18424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:20:40.557610 18424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:20:40.557708 18424 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:20:40.557834 18424 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:20:40.571133 18424 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:35297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:40565,127.17.110.61:35929,127.17.110.60:34417
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.62.keytab
--principal=kudu/127.17.110.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.62:40565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.110.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:20:40.573133 18424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:20:40.574839 18424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:20:40.592593 18430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:40.592509 18434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:40.592550 18431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:20:40.594365 18424 server_base.cc:1034] running on GCE node
I20260807 18:20:40.595450 18424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:20:40.597858 18424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:20:40.599220 18424 hybrid_clock.cc:648] HybridClock initialized: now 1786126840599089 us; error 80 us; skew 500 ppm
I20260807 18:20:40.599684 18424 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:20:40 dist-test-slave-39l8 krb5kdc[18407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126840, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:20:40.603588 18424 init.cc:377] Logged in from keytab as kudu/127.17.110.62@KRBTEST.COM (short username kudu)
I20260807 18:20:40.605401 18424 webserver.cc:460] Webserver started at http://127.17.110.62:46753/ using document root <none> and password file <none>
I20260807 18:20:40.606215 18424 fs_manager.cc:359] Metadata directory not provided
I20260807 18:20:40.606366 18424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:20:40.606791 18424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:20:40.610193 18424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-0/data/instance:
uuid: "7881b29d47df4d188edfd8d10a1c8c35"
format_stamp: "Formatted at 2026-08-07 18:20:40 on dist-test-slave-39l8"
server_key: "c6eff4e1e014db4d70fb5b6aa0fd35b6"
server_key_iv: "a12d9f9869ed507556b2d9c1248b79ba"
server_key_version: "encryptionkey@0"
I20260807 18:20:40.611246 18424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-0/wal/instance:
uuid: "7881b29d47df4d188edfd8d10a1c8c35"
format_stamp: "Formatted at 2026-08-07 18:20:40 on dist-test-slave-39l8"
server_key: "c6eff4e1e014db4d70fb5b6aa0fd35b6"
server_key_iv: "a12d9f9869ed507556b2d9c1248b79ba"
server_key_version: "encryptionkey@0"
I20260807 18:20:40.617671 18424 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.001s
I20260807 18:20:40.622042 18442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:40.623844 18424 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 18:20:40.624107 18424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-0/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-0/wal
uuid: "7881b29d47df4d188edfd8d10a1c8c35"
format_stamp: "Formatted at 2026-08-07 18:20:40 on dist-test-slave-39l8"
server_key: "c6eff4e1e014db4d70fb5b6aa0fd35b6"
server_key_iv: "a12d9f9869ed507556b2d9c1248b79ba"
server_key_version: "encryptionkey@0"
I20260807 18:20:40.624388 18424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:20:40.655542 18424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:20:40.661883 18424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:20:40.662424 18424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:20:40.687058 18424 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.62:40565
I20260807 18:20:40.687233 18520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.62:40565 every 8 connection(s)
I20260807 18:20:40.688995 18424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-0/data/info.pb
I20260807 18:20:40.694880 18523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:20:40.698843 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 18424
I20260807 18:20:40.699107 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-0/wal/instance
I20260807 18:20:40.699585 17848 external_mini_cluster.cc:1428] Setting key ecc5decbca3ef1675ad171408ad71f9c
I20260807 18:20:40.703145 18523 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40565 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:20:40 dist-test-slave-39l8 krb5kdc[18407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126840, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:20:40.737450 18526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:40.707174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40565 (local address 127.0.0.1:48266)
0807 18:20:40.708203 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:40.708260 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 18:20:40.711107 (+ 2847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:20:40.712754 (+ 1647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:20:40.712765 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:20:40.713773 (+ 1008us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:20:40.714436 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:40.714475 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:40.717305 (+ 2830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:40.717311 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:40.719702 (+ 2391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:40.719733 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:40.721662 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:40.721673 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:40.722005 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:40.723022 (+ 1017us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:20:40.723059 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:20:40.726904 (+ 3845us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:20:40.732392 (+ 5488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:40.732406 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:40.732421 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:40.732942 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:40.733473 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:40.733480 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:40.733483 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:40.733675 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:40.734618 (+ 943us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:20:40.734629 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:20:40.734930 (+ 301us) client_negotiation.cc:769] Sending connection context
0807 18:20:40.735213 (+ 283us) client_negotiation.cc:240] Negotiation successful
0807 18:20:40.735940 (+ 727us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"mutex_wait_us":210,"thread_start_us":464,"threads_started":1}
I20260807 18:20:40.737470 18527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:40.708846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48266 (local address 127.17.110.62:40565)
0807 18:20:40.709810 (+ 964us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:40.709819 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:40.709887 (+ 68us) server_negotiation.cc:407] Connection header received
0807 18:20:40.711843 (+ 1956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:40.711852 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:40.712400 (+ 548us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:40.712646 (+ 246us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:40.714950 (+ 2304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:40.716984 (+ 2034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:40.719945 (+ 2961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:40.721247 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:40.721553 (+ 306us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:40.727223 (+ 5670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:40.727236 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:40.727257 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:40.727303 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:40.732162 (+ 4859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:40.733121 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:40.733131 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:40.733135 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:40.733334 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:40.733816 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:40.733821 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:40.733824 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:40.734448 (+ 624us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:40.734736 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:40.735364 (+ 628us) server_negotiation.cc:299] Negotiation successful
0807 18:20:40.735803 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":690,"thread_start_us":475,"threads_started":1}
WARNING: no policy specified for kudu/127.17.110.61@KRBTEST.COM; defaulting to no policy
I20260807 18:20:40.741019 18523 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35929 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:20:40.742482 18526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:40.741938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:35929 (local address 127.0.0.1:50262)
0807 18:20:40.742151 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:40.742287 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.61:35929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260807 18:20:40.742965 18445 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.110.61:35929: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.61@KRBTEST.COM" created.
I20260807 18:20:40.744796 18526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:40.744356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:35929 (local address 127.0.0.1:50274)
0807 18:20:40.744552 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:40.744642 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.61:35929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260807 18:20:40.745020 18523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.61:35929: Network error: Client connection negotiation failed: client connection to 127.17.110.61:35929: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.61.keytab.
Entry for principal kudu/127.17.110.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.61.keytab.
I20260807 18:20:40.790516 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.61:35929
--webserver_interface=127.17.110.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.61.keytab
--principal=kudu/127.17.110.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:35297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:40565,127.17.110.61:35929,127.17.110.60:34417
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:20:40.794459 18523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35929 } attempt: 1
I20260807 18:20:40.796159 18526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:40.795665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:35929 (local address 127.0.0.1:50284)
0807 18:20:40.795883 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:40.796001 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.61:35929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 18:20:40.797327 18526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:40.796917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:35929 (local address 127.0.0.1:50288)
0807 18:20:40.797083 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:40.797171 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.61:35929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 18:20:40.797538 18523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.61:35929: Network error: Client connection negotiation failed: client connection to 127.17.110.61:35929: connect: Connection refused (error 111)
I20260807 18:20:40.865871 18523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35929 } attempt: 2
I20260807 18:20:40.867260 18526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:40.866661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:35929 (local address 127.0.0.1:50292)
0807 18:20:40.867010 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:40.867111 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.61:35929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
I20260807 18:20:40.868366 18526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:40.867956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:35929 (local address 127.0.0.1:50296)
0807 18:20:40.868125 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:40.868215 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.61:35929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 18:20:40.868577 18523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.61:35929: Network error: Client connection negotiation failed: client connection to 127.17.110.61:35929: connect: Connection refused (error 111)
I20260807 18:20:40.959900 18523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35929 } attempt: 3
I20260807 18:20:40.961377 18526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:40.960850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:35929 (local address 127.0.0.1:50302)
0807 18:20:40.961091 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:40.961198 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.61:35929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260807 18:20:40.962618 18526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:40.962178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:35929 (local address 127.0.0.1:50312)
0807 18:20:40.962356 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:40.962460 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.61:35929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260807 18:20:40.962872 18523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.61:35929: Network error: Client connection negotiation failed: client connection to 127.17.110.61:35929: connect: Connection refused (error 111)
I20260807 18:20:41.106205 18523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35929 } attempt: 4
I20260807 18:20:41.107730 18526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.107082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:35929 (local address 127.0.0.1:50322)
0807 18:20:41.107475 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:41.107572 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.61:35929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216}
I20260807 18:20:41.108990 18526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.108539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:35929 (local address 127.0.0.1:50330)
0807 18:20:41.108727 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:41.108841 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.61:35929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260807 18:20:41.109284 18523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.61:35929: Network error: Client connection negotiation failed: client connection to 127.17.110.61:35929: connect: Connection refused (error 111)
W20260807 18:20:41.170869 18531 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:20:41.171316 18531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:20:41.171449 18531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:20:41.171587 18531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:20:41.179522 18531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:20:41.179672 18531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:20:41.179796 18531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:20:41.179893 18531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:20:41.180006 18531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:20:41.191022 18531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:35297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:40565,127.17.110.61:35929,127.17.110.60:34417
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.61.keytab
--principal=kudu/127.17.110.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.61:35929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.110.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:20:41.192538 18531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:20:41.193948 18531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:20:41.207468 18543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:41.207563 18541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:41.207507 18539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:20:41.212793 18531 server_base.cc:1034] running on GCE node
I20260807 18:20:41.213855 18531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:20:41.215762 18531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:20:41.217067 18531 hybrid_clock.cc:648] HybridClock initialized: now 1786126841216920 us; error 132 us; skew 500 ppm
I20260807 18:20:41.217479 18531 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:20:41 dist-test-slave-39l8 krb5kdc[18407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126841, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:20:41.221477 18531 init.cc:377] Logged in from keytab as kudu/127.17.110.61@KRBTEST.COM (short username kudu)
I20260807 18:20:41.223649 18531 webserver.cc:460] Webserver started at http://127.17.110.61:38661/ using document root <none> and password file <none>
I20260807 18:20:41.224515 18531 fs_manager.cc:359] Metadata directory not provided
I20260807 18:20:41.224663 18531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:20:41.225024 18531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:20:41.228500 18531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-1/data/instance:
uuid: "2eb638ca160c415aa129ed6fed171acd"
format_stamp: "Formatted at 2026-08-07 18:20:41 on dist-test-slave-39l8"
server_key: "b76c6829c06bf5f31a5ac230378c43dd"
server_key_iv: "2b463aa3a07dffe1b8e3e1f162ca1c29"
server_key_version: "encryptionkey@0"
I20260807 18:20:41.229606 18531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-1/wal/instance:
uuid: "2eb638ca160c415aa129ed6fed171acd"
format_stamp: "Formatted at 2026-08-07 18:20:41 on dist-test-slave-39l8"
server_key: "b76c6829c06bf5f31a5ac230378c43dd"
server_key_iv: "2b463aa3a07dffe1b8e3e1f162ca1c29"
server_key_version: "encryptionkey@0"
I20260807 18:20:41.236048 18531 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 18:20:41.240633 18556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:41.242349 18531 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:20:41.242596 18531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-1/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-1/wal
uuid: "2eb638ca160c415aa129ed6fed171acd"
format_stamp: "Formatted at 2026-08-07 18:20:41 on dist-test-slave-39l8"
server_key: "b76c6829c06bf5f31a5ac230378c43dd"
server_key_iv: "2b463aa3a07dffe1b8e3e1f162ca1c29"
server_key_version: "encryptionkey@0"
I20260807 18:20:41.242916 18531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:20:41.270205 18531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:20:41.276573 18531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:20:41.276963 18531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:20:41.302603 18531 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.61:35929
I20260807 18:20:41.302649 18621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.61:35929 every 8 connection(s)
I20260807 18:20:41.304311 18531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-1/data/info.pb
I20260807 18:20:41.310128 18622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:20:41.312757 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 18531
I20260807 18:20:41.313009 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-1/wal/instance
I20260807 18:20:41.313512 17848 external_mini_cluster.cc:1428] Setting key 9d464203ea41dfd93070e81a1da669f7
I20260807 18:20:41.317836 18622 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40565 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:20:41 dist-test-slave-39l8 krb5kdc[18407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126841, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.61@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:20:41.343397 18625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.320843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48164 (local address 127.17.110.62:40565)
0807 18:20:41.321441 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:41.321446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:41.322701 (+ 1255us) server_negotiation.cc:407] Connection header received
0807 18:20:41.324977 (+ 2276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:41.324983 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:41.325148 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:41.325283 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:41.327167 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:41.327953 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:41.329963 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:41.330617 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:41.330836 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:41.335829 (+ 4993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:41.335837 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:41.335840 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:41.335886 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:41.339330 (+ 3444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:41.340402 (+ 1072us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:41.340407 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:41.340410 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:41.340569 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:41.341148 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:41.341152 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:41.341155 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:41.341490 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:41.341710 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:41.342614 (+ 904us) server_negotiation.cc:299] Negotiation successful
0807 18:20:41.342951 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":347,"threads_started":1}
I20260807 18:20:41.344666 18627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.321619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40565 (local address 127.0.0.1:48164)
0807 18:20:41.322508 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:41.322551 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 18:20:41.324603 (+ 2052us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:20:41.325592 (+ 989us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:20:41.325602 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:20:41.326204 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:20:41.326907 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:41.326931 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:41.328147 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:41.328151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:41.329718 (+ 1567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:41.329739 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:41.330957 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:41.330968 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:41.331360 (+ 392us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:41.332233 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:20:41.332270 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:20:41.335494 (+ 3224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:20:41.339567 (+ 4073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:41.339580 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:41.339610 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:41.340212 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:41.340721 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:41.340728 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:41.340732 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:41.340968 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:41.341677 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:20:41.341688 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:20:41.342154 (+ 466us) client_negotiation.cc:769] Sending connection context
0807 18:20:41.342482 (+ 328us) client_negotiation.cc:240] Negotiation successful
0807 18:20:41.343152 (+ 670us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":371,"threads_started":1}
I20260807 18:20:41.346949 18622 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35929 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.17.110.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.110.60@KRBTEST.COM" created.
Aug 07 18:20:41 dist-test-slave-39l8 krb5kdc[18407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126841, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.61@KRBTEST.COM for kudu/127.17.110.61@KRBTEST.COM
I20260807 18:20:41.368173 18630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.348157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38264 (local address 127.17.110.61:35929)
0807 18:20:41.348990 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:41.348999 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:41.349040 (+ 41us) server_negotiation.cc:407] Connection header received
0807 18:20:41.349110 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:41.349126 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:41.349349 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:41.349536 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:41.350568 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:41.352115 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:41.353623 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:41.354903 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:41.355083 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:41.359411 (+ 4328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:41.359436 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:41.359462 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:41.359514 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:41.364037 (+ 4523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:41.365239 (+ 1202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:41.365252 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:41.365259 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:41.365433 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:41.365980 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:41.365988 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:41.365994 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:41.366545 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:41.366816 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:41.367385 (+ 569us) server_negotiation.cc:299] Negotiation successful
0807 18:20:41.367800 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":147,"server-negotiator.queue_time_us":610,"thread_start_us":409,"threads_started":1}
I20260807 18:20:41.368256 18627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.348157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:35929 (local address 127.0.0.1:38264)
0807 18:20:41.348435 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:41.348457 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:20:41.348608 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:20:41.349584 (+ 976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:20:41.349590 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:20:41.350141 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:20:41.350330 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:41.350341 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:41.352279 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:41.352289 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:41.353343 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:41.353360 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:41.355090 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:41.355096 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:41.355277 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:41.356022 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:20:41.356042 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:20:41.359078 (+ 3036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:20:41.364276 (+ 5198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:41.364286 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:41.364294 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:41.365023 (+ 729us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:41.365609 (+ 586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:41.365618 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:41.365625 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:41.365841 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:41.366750 (+ 909us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:20:41.366759 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:20:41.366994 (+ 235us) client_negotiation.cc:769] Sending connection context
0807 18:20:41.367193 (+ 199us) client_negotiation.cc:240] Negotiation successful
0807 18:20:41.367637 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 18:20:41.371611 18622 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34417 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:20:41.373144 18627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.372526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:34417 (local address 127.0.0.1:43932)
0807 18:20:41.372769 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:41.372897 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:34417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260807 18:20:41.373452 18558 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.110.60:34417: connect: Connection refused (error 111)
I20260807 18:20:41.375401 18627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.374947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:34417 (local address 127.0.0.1:43938)
0807 18:20:41.375174 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:41.375267 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:34417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 18:20:41.375641 18622 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.60:34417: Network error: Client connection negotiation failed: client connection to 127.17.110.60:34417: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.110.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.60.keytab.
Entry for principal kudu/127.17.110.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.60.keytab.
I20260807 18:20:41.398205 17848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
/tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.60:34417
--webserver_interface=127.17.110.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.60.keytab
--principal=kudu/127.17.110.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.110.20:35297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.110.62:40565,127.17.110.61:35929,127.17.110.60:34417
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:20:41.408550 18523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35929 } attempt: 5
Aug 07 18:20:41 dist-test-slave-39l8 krb5kdc[18407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126840, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for kudu/127.17.110.61@KRBTEST.COM
I20260807 18:20:41.424993 18622 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34417 } attempt: 1
I20260807 18:20:41.426268 18526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.409388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:35929 (local address 127.0.0.1:38270)
0807 18:20:41.409685 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:41.409711 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 18:20:41.409881 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:20:41.410371 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:20:41.410378 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:20:41.410999 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:20:41.411289 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:41.411306 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:41.412575 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:41.412581 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:41.413487 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:41.413500 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:41.414607 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:41.414612 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:41.414872 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:41.415635 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:20:41.415654 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:20:41.418015 (+ 2361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:20:41.422644 (+ 4629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:41.422650 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:41.422653 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:41.423307 (+ 654us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:41.423866 (+ 559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:41.423870 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:41.423872 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:41.424034 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:41.424913 (+ 879us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:20:41.424918 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:20:41.425234 (+ 316us) client_negotiation.cc:769] Sending connection context
0807 18:20:41.425416 (+ 182us) client_negotiation.cc:240] Negotiation successful
0807 18:20:41.425836 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110}
I20260807 18:20:41.426381 18627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.425766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:34417 (local address 127.0.0.1:43940)
0807 18:20:41.426074 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:41.426199 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:34417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172}
I20260807 18:20:41.426331 18630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.409480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38270 (local address 127.17.110.61:35929)
0807 18:20:41.409679 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:41.409709 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:41.409802 (+ 93us) server_negotiation.cc:407] Connection header received
0807 18:20:41.410104 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:41.410109 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:41.410187 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:41.410288 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:41.411481 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:41.412402 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:41.413660 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:41.414435 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:41.414591 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:41.418288 (+ 3697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:41.418297 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:41.418305 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:41.418359 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:41.422411 (+ 4052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:41.423481 (+ 1070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:41.423491 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:41.423497 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:41.423673 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:41.424231 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:41.424240 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:41.424246 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:41.424721 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:41.424916 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:41.425597 (+ 681us) server_negotiation.cc:299] Negotiation successful
0807 18:20:41.425892 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 18:20:41.427893 18627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.427338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:34417 (local address 127.0.0.1:43944)
0807 18:20:41.427547 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:41.427689 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:34417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260807 18:20:41.428213 18622 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.60:34417: Network error: Client connection negotiation failed: client connection to 127.17.110.60:34417: connect: Connection refused (error 111)
I20260807 18:20:41.428321 18523 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34417 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:20:41.429652 18526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.429085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:34417 (local address 127.0.0.1:43954)
0807 18:20:41.429381 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:41.429471 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:34417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260807 18:20:41.431170 18526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.430577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:34417 (local address 127.0.0.1:43956)
0807 18:20:41.430802 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:41.430919 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:34417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 18:20:41.431509 18523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.60:34417: Network error: Client connection negotiation failed: client connection to 127.17.110.60:34417: connect: Connection refused (error 111)
I20260807 18:20:41.482787 18523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34417 } attempt: 1
I20260807 18:20:41.484252 18526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.483678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:34417 (local address 127.0.0.1:43964)
0807 18:20:41.483948 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:41.484055 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:34417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 18:20:41.485455 18526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.485042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:34417 (local address 127.0.0.1:43972)
0807 18:20:41.485250 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:41.485336 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:34417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260807 18:20:41.485718 18523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.60:34417: Network error: Client connection negotiation failed: client connection to 127.17.110.60:34417: connect: Connection refused (error 111)
I20260807 18:20:41.496495 18622 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34417 } attempt: 2
I20260807 18:20:41.498059 18627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.497332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:34417 (local address 127.0.0.1:43980)
0807 18:20:41.497630 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:41.497795 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:34417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260807 18:20:41.499639 18627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.499092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:34417 (local address 127.0.0.1:43988)
0807 18:20:41.499307 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:41.499429 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:34417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260807 18:20:41.500005 18622 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.60:34417: Network error: Client connection negotiation failed: client connection to 127.17.110.60:34417: connect: Connection refused (error 111)
I20260807 18:20:41.553972 18523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34417 } attempt: 2
I20260807 18:20:41.555312 18526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.554808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:34417 (local address 127.0.0.1:44002)
0807 18:20:41.555035 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:41.555132 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:34417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 18:20:41.556684 18526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.556231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:34417 (local address 127.0.0.1:44010)
0807 18:20:41.556436 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:41.556537 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:34417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260807 18:20:41.556950 18523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.60:34417: Network error: Client connection negotiation failed: client connection to 127.17.110.60:34417: connect: Connection refused (error 111)
I20260807 18:20:41.591300 18622 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34417 } attempt: 3
I20260807 18:20:41.592730 18627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.592179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:34417 (local address 127.0.0.1:44020)
0807 18:20:41.592429 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:41.592564 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:34417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260807 18:20:41.594089 18627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.593550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:34417 (local address 127.0.0.1:44034)
0807 18:20:41.593854 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:41.593946 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:34417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171}
W20260807 18:20:41.594336 18622 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.60:34417: Network error: Client connection negotiation failed: client connection to 127.17.110.60:34417: connect: Connection refused (error 111)
I20260807 18:20:41.663197 18523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34417 } attempt: 3
I20260807 18:20:41.664520 18526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.663974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:34417 (local address 127.0.0.1:44044)
0807 18:20:41.664202 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:41.664329 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:34417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 18:20:41.665939 18526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.665430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:34417 (local address 127.0.0.1:44050)
0807 18:20:41.665715 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:41.665805 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:34417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
W20260807 18:20:41.666199 18523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.60:34417: Network error: Client connection negotiation failed: client connection to 127.17.110.60:34417: connect: Connection refused (error 111)
I20260807 18:20:41.737643 18622 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34417 } attempt: 4
I20260807 18:20:41.739225 18627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.738526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:34417 (local address 127.0.0.1:44058)
0807 18:20:41.738937 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:41.739047 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:34417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260807 18:20:41.740377 18627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.739996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:34417 (local address 127.0.0.1:44066)
0807 18:20:41.740174 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:41.740253 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:34417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 18:20:41.740662 18622 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.60:34417: Network error: Client connection negotiation failed: client connection to 127.17.110.60:34417: connect: Connection refused (error 111)
W20260807 18:20:41.789702 18636 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:20:41.790122 18636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:20:41.790273 18636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:20:41.790405 18636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:20:41.798242 18636 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:20:41.798372 18636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:20:41.798430 18636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:20:41.798547 18636 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:20:41.798659 18636 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:20:41.809878 18636 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.110.20:35297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.110.62:40565,127.17.110.61:35929,127.17.110.60:34417
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/krb5kdc/kudu_127.17.110.60.keytab
--principal=kudu/127.17.110.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.110.60:34417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.110.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 17:13:57 UTC on ccycloud.quasar-mgcilv.root.comops.site
build id 4572599
ASAN enabled
I20260807 18:20:41.811581 18636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:20:41.813186 18636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:20:41.827435 18644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:41.827355 18645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:20:41.827463 18647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:20:41.828647 18636 server_base.cc:1034] running on GCE node
I20260807 18:20:41.829696 18636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:20:41.831632 18636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:20:41.832937 18636 hybrid_clock.cc:648] HybridClock initialized: now 1786126841832837 us; error 84 us; skew 500 ppm
I20260807 18:20:41.833339 18636 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:20:41 dist-test-slave-39l8 krb5kdc[18407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126841, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:20:41.838474 18636 init.cc:377] Logged in from keytab as kudu/127.17.110.60@KRBTEST.COM (short username kudu)
I20260807 18:20:41.840662 18636 webserver.cc:460] Webserver started at http://127.17.110.60:36001/ using document root <none> and password file <none>
I20260807 18:20:41.841533 18636 fs_manager.cc:359] Metadata directory not provided
I20260807 18:20:41.841660 18636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:20:41.841985 18636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:20:41.843488 18523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34417 } attempt: 4
I20260807 18:20:41.844772 18526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.844318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:34417 (local address 127.0.0.1:44080)
0807 18:20:41.844526 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:41.844626 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:34417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 18:20:41.846045 18526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.845619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:34417 (local address 127.0.0.1:44096)
0807 18:20:41.845817 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:41.845909 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.110.60:34417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260807 18:20:41.846027 18636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-2/data/instance:
uuid: "3a1be66c26f6486b8969d0da789d9503"
format_stamp: "Formatted at 2026-08-07 18:20:41 on dist-test-slave-39l8"
server_key: "ec2430c92466c3076b559ec495f435b2"
server_key_iv: "bb90e8eb9dd190dd0d7a93346714604f"
server_key_version: "encryptionkey@0"
W20260807 18:20:41.846390 18523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.110.60:34417: Network error: Client connection negotiation failed: client connection to 127.17.110.60:34417: connect: Connection refused (error 111)
I20260807 18:20:41.847122 18636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-2/wal/instance:
uuid: "3a1be66c26f6486b8969d0da789d9503"
format_stamp: "Formatted at 2026-08-07 18:20:41 on dist-test-slave-39l8"
server_key: "ec2430c92466c3076b559ec495f435b2"
server_key_iv: "bb90e8eb9dd190dd0d7a93346714604f"
server_key_version: "encryptionkey@0"
I20260807 18:20:41.854498 18636 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260807 18:20:41.859261 18654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:20:41.861440 18636 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:20:41.861793 18636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-2/data,/tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-2/wal
uuid: "3a1be66c26f6486b8969d0da789d9503"
format_stamp: "Formatted at 2026-08-07 18:20:41 on dist-test-slave-39l8"
server_key: "ec2430c92466c3076b559ec495f435b2"
server_key_iv: "bb90e8eb9dd190dd0d7a93346714604f"
server_key_version: "encryptionkey@0"
I20260807 18:20:41.862107 18636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:20:41.881142 18636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:20:41.887159 18636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:20:41.887516 18636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:20:41.910481 18636 rpc_server.cc:307] RPC server started. Bound to: 127.17.110.60:34417
I20260807 18:20:41.910530 18726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.110.60:34417 every 8 connection(s)
I20260807 18:20:41.912194 18636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-2/data/info.pb
I20260807 18:20:41.917148 18728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:20:41.922497 17848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu as pid 18636
I20260807 18:20:41.922787 17848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDgKS9Z/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126429568977-17848-0/minicluster-data/master-2/wal/instance
I20260807 18:20:41.923421 17848 external_mini_cluster.cc:1428] Setting key c60e1ae30e4ce92d417fb4eebfde1f98
I20260807 18:20:41.926003 18728 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40565 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:20:41 dist-test-slave-39l8 krb5kdc[18407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126841, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.60@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
Aug 07 18:20:41 dist-test-slave-39l8 krb5kdc[18407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.60@KRBTEST.COM
I20260807 18:20:41.953217 18739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.929752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48180 (local address 127.17.110.62:40565)
0807 18:20:41.931582 (+ 1830us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:41.931587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:41.931604 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:20:41.931664 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:41.931668 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:41.931730 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:41.931830 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:41.933829 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:41.935008 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:41.938665 (+ 3657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:41.939432 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:41.939613 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:41.945597 (+ 5984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:41.945609 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:41.945617 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:41.945682 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:41.949077 (+ 3395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:41.950134 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:41.950139 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:41.950141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:41.950292 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:41.950889 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:41.950894 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:41.950897 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:41.951315 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:41.951483 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:41.952588 (+ 1105us) server_negotiation.cc:299] Negotiation successful
0807 18:20:41.952803 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1687,"thread_start_us":1411,"threads_started":1}
I20260807 18:20:41.954478 18733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.929646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.62:40565 (local address 127.0.0.1:48180)
0807 18:20:41.930970 (+ 1324us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:41.931151 (+ 181us) client_negotiation.cc:174] Beginning negotiation
0807 18:20:41.931350 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:20:41.931978 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:20:41.931989 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:20:41.932966 (+ 977us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:20:41.933591 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:41.933615 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:41.935253 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:41.935258 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:41.938290 (+ 3032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:41.938316 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:41.939824 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:41.939832 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:41.940155 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:41.941248 (+ 1093us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:20:41.941296 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:20:41.945162 (+ 3866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:20:41.949252 (+ 4090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:41.949265 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:41.949281 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:41.949964 (+ 683us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:41.950438 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:41.950443 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:41.950447 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:41.950761 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:41.951487 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:20:41.951495 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:20:41.952033 (+ 538us) client_negotiation.cc:769] Sending connection context
0807 18:20:41.952406 (+ 373us) client_negotiation.cc:240] Negotiation successful
0807 18:20:41.953020 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":825,"mutex_wait_us":55,"thread_start_us":367,"threads_started":1}
I20260807 18:20:41.955078 18732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.925783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44100 (local address 127.17.110.60:34417)
0807 18:20:41.926859 (+ 1076us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:41.926873 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:41.926924 (+ 51us) server_negotiation.cc:407] Connection header received
0807 18:20:41.929382 (+ 2458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:41.929419 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:41.930150 (+ 731us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:41.930839 (+ 689us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:41.933531 (+ 2692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:41.935424 (+ 1893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:41.938642 (+ 3218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:41.939839 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:41.940107 (+ 268us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:41.946662 (+ 6555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:41.946681 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:41.946732 (+ 51us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:41.946796 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:41.950667 (+ 3871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:41.951739 (+ 1072us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:41.951749 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:41.951752 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:41.951923 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:41.952393 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:41.952398 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:41.952401 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:41.952869 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:41.953193 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:41.954212 (+ 1019us) server_negotiation.cc:299] Negotiation successful
0807 18:20:41.954425 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":8,"server-negotiator.queue_time_us":749,"thread_start_us":484,"threads_started":1}
I20260807 18:20:41.956898 18728 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35929 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:20:41 dist-test-slave-39l8 krb5kdc[18407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126841, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.60@KRBTEST.COM for kudu/127.17.110.61@KRBTEST.COM
Aug 07 18:20:41 dist-test-slave-39l8 krb5kdc[18407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.62@KRBTEST.COM
I20260807 18:20:41.973953 18733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.958470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.61:35929 (local address 127.0.0.1:38282)
0807 18:20:41.958858 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:41.958879 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:20:41.959291 (+ 412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:20:41.959709 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:20:41.959713 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:20:41.960145 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:20:41.960306 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:41.960315 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:41.961521 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:41.961527 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:41.962501 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:41.962514 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:41.963319 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:41.963326 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:41.963547 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:41.964338 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:20:41.964368 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:20:41.967211 (+ 2843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:20:41.971157 (+ 3946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:41.971164 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:41.971168 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:41.971633 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:41.972120 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:41.972126 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:41.972130 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:41.972311 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:41.972868 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:20:41.972873 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:20:41.973047 (+ 174us) client_negotiation.cc:769] Sending connection context
0807 18:20:41.973213 (+ 166us) client_negotiation.cc:240] Negotiation successful
0807 18:20:41.973549 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240}
I20260807 18:20:41.974015 18740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.958604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38282 (local address 127.17.110.61:35929)
0807 18:20:41.959302 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:41.959309 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:41.959333 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:20:41.959430 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:41.959435 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:41.959524 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:41.959677 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:41.960518 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:41.961362 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:41.962672 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:41.963141 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:41.963308 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:41.967497 (+ 4189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:41.967503 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:41.967506 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:41.967548 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:41.970971 (+ 3423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:41.971799 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:41.971806 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:41.971810 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:41.971968 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:41.972437 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:41.972443 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:41.972448 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:41.972736 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:41.972905 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:41.973324 (+ 419us) server_negotiation.cc:299] Negotiation successful
0807 18:20:41.973553 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":393,"threads_started":1}
I20260807 18:20:41.975534 18739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.959119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48186 (local address 127.17.110.62:40565)
0807 18:20:41.959313 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:41.959318 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:41.959342 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:20:41.959421 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:41.959427 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:41.959510 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:41.959642 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:41.961008 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:41.961735 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:41.963147 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:41.963647 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:41.963818 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:41.969127 (+ 5309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:41.969134 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:41.969138 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:41.969183 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:41.972028 (+ 2845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:41.972851 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:41.972855 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:41.972858 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:41.973001 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:41.973438 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:41.973442 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:41.973444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:41.973736 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:41.974045 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:41.974953 (+ 908us) server_negotiation.cc:299] Negotiation successful
0807 18:20:41.975204 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 18:20:41.975804 18728 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34417 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:20:41 dist-test-slave-39l8 krb5kdc[18407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126841, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.60@KRBTEST.COM for kudu/127.17.110.60@KRBTEST.COM
Aug 07 18:20:41 dist-test-slave-39l8 krb5kdc[18407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.110.61@KRBTEST.COM
I20260807 18:20:41.992698 18732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.977199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44102 (local address 127.17.110.60:34417)
0807 18:20:41.977362 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:41.977367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:41.977382 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:20:41.977427 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:41.977430 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:41.977486 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:41.977601 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:41.978491 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:41.979285 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:41.980510 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:41.980929 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:41.981073 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:41.984648 (+ 3575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:41.984658 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:41.984705 (+ 47us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:41.984760 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:41.989430 (+ 4670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:41.990345 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:41.990350 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:41.990353 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:41.990529 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:41.991029 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:41.991033 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:41.991035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:41.991373 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:41.991558 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:41.991980 (+ 422us) server_negotiation.cc:299] Negotiation successful
0807 18:20:41.992266 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:20:41.992833 18733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.976826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:34417 (local address 127.0.0.1:44102)
0807 18:20:41.977054 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:41.977072 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:20:41.977217 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:20:41.977618 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:20:41.977622 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:20:41.978182 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:20:41.978345 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:41.978354 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:41.979449 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:41.979458 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:41.980355 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:41.980369 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:41.981099 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:41.981105 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:41.981308 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:41.982033 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:20:41.982052 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:20:41.984384 (+ 2332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:20:41.989705 (+ 5321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:41.989710 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:41.989714 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:41.990189 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:41.990689 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:41.990694 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:41.990698 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:41.990892 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:41.991514 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:20:41.991519 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:20:41.991696 (+ 177us) client_negotiation.cc:769] Sending connection context
0807 18:20:41.991881 (+ 185us) client_negotiation.cc:240] Negotiation successful
0807 18:20:41.992433 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":169}
I20260807 18:20:41.997715 18740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:41.977986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38296 (local address 127.17.110.61:35929)
0807 18:20:41.978230 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:41.978234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:41.978252 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:20:41.978302 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:41.978306 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:41.978369 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:41.978509 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:41.979988 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:41.980720 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:41.982153 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:41.982601 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:41.982860 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:41.990449 (+ 7589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:41.990455 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:41.990458 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:41.990498 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:41.994031 (+ 3533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:41.995228 (+ 1197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:41.995350 (+ 122us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:41.995354 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:41.995537 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:41.996067 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:41.996072 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:41.996074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:41.996403 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:41.996740 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:41.997067 (+ 327us) server_negotiation.cc:299] Negotiation successful
0807 18:20:41.997342 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260807 18:20:41.999675 18728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a1be66c26f6486b8969d0da789d9503: Bootstrap starting.
I20260807 18:20:42.003484 17848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:20:42.004112 18728 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a1be66c26f6486b8969d0da789d9503: Neither blocks nor log segments found. Creating new log.
I20260807 18:20:42.005517 18728 log.cc:824] T 00000000000000000000000000000000 P 3a1be66c26f6486b8969d0da789d9503: Log is configured to *not* fsync() on all Append() calls
I20260807 18:20:42.006245 18488 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "7881b29d47df4d188edfd8d10a1c8c35"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48186
I20260807 18:20:42.009738 18728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a1be66c26f6486b8969d0da789d9503: No bootstrap required, opened a new log
I20260807 18:20:42.020852 18728 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a1be66c26f6486b8969d0da789d9503 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7881b29d47df4d188edfd8d10a1c8c35" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40565 } } peers { permanent_uuid: "2eb638ca160c415aa129ed6fed171acd" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35929 } } peers { permanent_uuid: "3a1be66c26f6486b8969d0da789d9503" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34417 } }
I20260807 18:20:42.021341 18728 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a1be66c26f6486b8969d0da789d9503 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a1be66c26f6486b8969d0da789d9503, State: Initialized, Role: FOLLOWER
I20260807 18:20:42.021919 18728 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a1be66c26f6486b8969d0da789d9503 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7881b29d47df4d188edfd8d10a1c8c35" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40565 } } peers { permanent_uuid: "2eb638ca160c415aa129ed6fed171acd" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35929 } } peers { permanent_uuid: "3a1be66c26f6486b8969d0da789d9503" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34417 } }
I20260807 18:20:42.023450 18743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a1be66c26f6486b8969d0da789d9503 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7881b29d47df4d188edfd8d10a1c8c35" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40565 } } peers { permanent_uuid: "2eb638ca160c415aa129ed6fed171acd" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35929 } } peers { permanent_uuid: "3a1be66c26f6486b8969d0da789d9503" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34417 } } }
I20260807 18:20:42.023895 18743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a1be66c26f6486b8969d0da789d9503 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:20:42.025315 18728 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a1be66c26f6486b8969d0da789d9503 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 18:20:42.036710 18756 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3a1be66c26f6486b8969d0da789d9503: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:20:42.036953 18756 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:20:42.039973 18622 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34417 } attempt: 5
I20260807 18:20:42.034332 18636 master_runner.cc:186] Error getting master registration for 127.17.110.62:40565: OK, instance_id { permanent_uuid: "7881b29d47df4d188edfd8d10a1c8c35" instance_seqno: 1786126840680231 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:20:42.048938 18636 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 07 18:20:42 dist-test-slave-39l8 krb5kdc[18407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126841, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.61@KRBTEST.COM for kudu/127.17.110.60@KRBTEST.COM
I20260807 18:20:42.061218 18732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:42.040787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44112 (local address 127.17.110.60:34417)
0807 18:20:42.041395 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:42.041400 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:42.041414 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:20:42.041471 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:42.041475 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:42.041533 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:42.041660 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:42.042648 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:42.043553 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:42.045389 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:42.046447 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:42.046666 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:42.052324 (+ 5658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:42.052332 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:42.052337 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:42.052391 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:42.057627 (+ 5236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:42.058844 (+ 1217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:42.058851 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:42.058854 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:42.059087 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:42.059551 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:42.059561 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:42.059564 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:42.059946 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:42.060128 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:42.060578 (+ 450us) server_negotiation.cc:299] Negotiation successful
0807 18:20:42.060817 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462}
I20260807 18:20:42.061213 18627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:42.040728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:34417 (local address 127.0.0.1:44112)
0807 18:20:42.041031 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:42.041049 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:20:42.041155 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:20:42.041704 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:20:42.041711 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:20:42.042252 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:20:42.042473 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:42.042487 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:42.043744 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:42.043752 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:42.045202 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:42.045222 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:42.046692 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:42.046825 (+ 133us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:42.047156 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:42.048105 (+ 949us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:20:42.048140 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:20:42.051959 (+ 3819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:20:42.057903 (+ 5944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:42.057910 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:42.057915 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:42.058614 (+ 699us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:42.059229 (+ 615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:42.059235 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:42.059238 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:42.059410 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:42.060092 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:20:42.060097 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:20:42.060280 (+ 183us) client_negotiation.cc:769] Sending connection context
0807 18:20:42.060443 (+ 163us) client_negotiation.cc:240] Negotiation successful
0807 18:20:42.060797 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184}
I20260807 18:20:42.066951 18622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2eb638ca160c415aa129ed6fed171acd: Bootstrap starting.
I20260807 18:20:42.072099 18622 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2eb638ca160c415aa129ed6fed171acd: Neither blocks nor log segments found. Creating new log.
I20260807 18:20:42.073491 18622 log.cc:824] T 00000000000000000000000000000000 P 2eb638ca160c415aa129ed6fed171acd: Log is configured to *not* fsync() on all Append() calls
I20260807 18:20:42.077328 18622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2eb638ca160c415aa129ed6fed171acd: No bootstrap required, opened a new log
I20260807 18:20:42.089882 18622 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2eb638ca160c415aa129ed6fed171acd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7881b29d47df4d188edfd8d10a1c8c35" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40565 } } peers { permanent_uuid: "2eb638ca160c415aa129ed6fed171acd" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35929 } } peers { permanent_uuid: "3a1be66c26f6486b8969d0da789d9503" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34417 } }
I20260807 18:20:42.090479 18622 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2eb638ca160c415aa129ed6fed171acd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2eb638ca160c415aa129ed6fed171acd, State: Initialized, Role: FOLLOWER
I20260807 18:20:42.091177 18622 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2eb638ca160c415aa129ed6fed171acd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7881b29d47df4d188edfd8d10a1c8c35" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40565 } } peers { permanent_uuid: "2eb638ca160c415aa129ed6fed171acd" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35929 } } peers { permanent_uuid: "3a1be66c26f6486b8969d0da789d9503" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34417 } }
I20260807 18:20:42.093118 18760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2eb638ca160c415aa129ed6fed171acd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7881b29d47df4d188edfd8d10a1c8c35" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40565 } } peers { permanent_uuid: "2eb638ca160c415aa129ed6fed171acd" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35929 } } peers { permanent_uuid: "3a1be66c26f6486b8969d0da789d9503" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34417 } } }
I20260807 18:20:42.093755 18760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2eb638ca160c415aa129ed6fed171acd [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:20:42.094795 18622 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2eb638ca160c415aa129ed6fed171acd [sys.catalog]: configured and running, proceeding with master startup.
W20260807 18:20:42.108865 18773 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2eb638ca160c415aa129ed6fed171acd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:20:42.109092 18773 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:20:42.104534 18531 master_runner.cc:186] Error getting master registration for 127.17.110.62:40565: OK, instance_id { permanent_uuid: "7881b29d47df4d188edfd8d10a1c8c35" instance_seqno: 1786126840680231 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:20:42.119812 18531 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 18:20:42.123576 18523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34417 } attempt: 5
Aug 07 18:20:42 dist-test-slave-39l8 krb5kdc[18407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126840, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.110.62@KRBTEST.COM for kudu/127.17.110.60@KRBTEST.COM
W20260807 18:20:42.140364 18661 tablet.cc:2369] T 00000000000000000000000000000000 P 3a1be66c26f6486b8969d0da789d9503: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:20:42.146878 18526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:42.124703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.110.60:34417 (local address 127.0.0.1:44128)
0807 18:20:42.125109 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0807 18:20:42.125261 (+ 152us) client_negotiation.cc:174] Beginning negotiation
0807 18:20:42.125437 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:20:42.125938 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:20:42.125945 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:20:42.126575 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:20:42.126901 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:42.126919 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:42.128809 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:42.128818 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:42.130620 (+ 1802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:20:42.130656 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:20:42.132486 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:20:42.132496 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:20:42.132769 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:42.133588 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:20:42.133619 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:20:42.137298 (+ 3679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:20:42.142819 (+ 5521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:42.142827 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:42.142831 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:42.143519 (+ 688us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:42.144178 (+ 659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:20:42.144185 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:20:42.144190 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:20:42.144459 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:20:42.145551 (+ 1092us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:20:42.145558 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:20:42.145796 (+ 238us) client_negotiation.cc:769] Sending connection context
0807 18:20:42.145969 (+ 173us) client_negotiation.cc:240] Negotiation successful
0807 18:20:42.146331 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251}
I20260807 18:20:42.148240 18732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:42.124864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44128 (local address 127.17.110.60:34417)
0807 18:20:42.125287 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:42.125296 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:42.125331 (+ 35us) server_negotiation.cc:407] Connection header received
0807 18:20:42.125625 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:42.125631 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:42.125733 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:42.125888 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:42.127171 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:42.128531 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:42.130925 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:42.131991 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:42.132192 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:42.137659 (+ 5467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:42.137669 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:42.137675 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:42.137731 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:42.142484 (+ 4753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:42.143760 (+ 1276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:42.143768 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:42.143772 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:42.143997 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:42.144657 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:42.144663 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:42.144667 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:42.145102 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:42.145452 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:42.147491 (+ 2039us) server_negotiation.cc:299] Negotiation successful
0807 18:20:42.147807 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206}
I20260807 18:20:42.156113 18523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7881b29d47df4d188edfd8d10a1c8c35: Bootstrap starting.
I20260807 18:20:42.161321 18523 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7881b29d47df4d188edfd8d10a1c8c35: Neither blocks nor log segments found. Creating new log.
I20260807 18:20:42.163334 18523 log.cc:824] T 00000000000000000000000000000000 P 7881b29d47df4d188edfd8d10a1c8c35: Log is configured to *not* fsync() on all Append() calls
I20260807 18:20:42.168283 18523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7881b29d47df4d188edfd8d10a1c8c35: No bootstrap required, opened a new log
I20260807 18:20:42.178098 18523 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7881b29d47df4d188edfd8d10a1c8c35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7881b29d47df4d188edfd8d10a1c8c35" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40565 } } peers { permanent_uuid: "2eb638ca160c415aa129ed6fed171acd" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35929 } } peers { permanent_uuid: "3a1be66c26f6486b8969d0da789d9503" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34417 } }
I20260807 18:20:42.178817 18523 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7881b29d47df4d188edfd8d10a1c8c35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7881b29d47df4d188edfd8d10a1c8c35, State: Initialized, Role: FOLLOWER
I20260807 18:20:42.179589 18523 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7881b29d47df4d188edfd8d10a1c8c35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7881b29d47df4d188edfd8d10a1c8c35" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40565 } } peers { permanent_uuid: "2eb638ca160c415aa129ed6fed171acd" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35929 } } peers { permanent_uuid: "3a1be66c26f6486b8969d0da789d9503" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34417 } }
I20260807 18:20:42.188905 18523 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7881b29d47df4d188edfd8d10a1c8c35 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:20:42.189033 18488 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7881b29d47df4d188edfd8d10a1c8c35 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 18:20:42.189388 18488 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7881b29d47df4d188edfd8d10a1c8c35 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:20:42.193805 18488 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7881b29d47df4d188edfd8d10a1c8c35 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7881b29d47df4d188edfd8d10a1c8c35" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40565 } } peers { permanent_uuid: "2eb638ca160c415aa129ed6fed171acd" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35929 } } peers { permanent_uuid: "3a1be66c26f6486b8969d0da789d9503" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34417 } }
I20260807 18:20:42.196089 18776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7881b29d47df4d188edfd8d10a1c8c35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7881b29d47df4d188edfd8d10a1c8c35" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40565 } } peers { permanent_uuid: "2eb638ca160c415aa129ed6fed171acd" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35929 } } peers { permanent_uuid: "3a1be66c26f6486b8969d0da789d9503" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34417 } } }
I20260807 18:20:42.196777 18776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7881b29d47df4d188edfd8d10a1c8c35 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:20:42.199105 18488 leader_election.cc:288] T 00000000000000000000000000000000 P 7881b29d47df4d188edfd8d10a1c8c35 [CANDIDATE]: Term 1 election: Requested vote from peers 2eb638ca160c415aa129ed6fed171acd (127.17.110.61:35929), 3a1be66c26f6486b8969d0da789d9503 (127.17.110.60:34417)
I20260807 18:20:42.199445 18592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7881b29d47df4d188edfd8d10a1c8c35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2eb638ca160c415aa129ed6fed171acd"
I20260807 18:20:42.200068 18592 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2eb638ca160c415aa129ed6fed171acd [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:20:42.204257 18692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7881b29d47df4d188edfd8d10a1c8c35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3a1be66c26f6486b8969d0da789d9503"
I20260807 18:20:42.204905 18692 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a1be66c26f6486b8969d0da789d9503 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:20:42.209182 18692 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3a1be66c26f6486b8969d0da789d9503 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7881b29d47df4d188edfd8d10a1c8c35 in term 1.
I20260807 18:20:42.211273 18592 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2eb638ca160c415aa129ed6fed171acd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7881b29d47df4d188edfd8d10a1c8c35 in term 1.
I20260807 18:20:42.212355 18445 leader_election.cc:302] T 00000000000000000000000000000000 P 7881b29d47df4d188edfd8d10a1c8c35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2eb638ca160c415aa129ed6fed171acd, 7881b29d47df4d188edfd8d10a1c8c35; no voters:
I20260807 18:20:42.213152 18776 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7881b29d47df4d188edfd8d10a1c8c35 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:20:42.223809 18776 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7881b29d47df4d188edfd8d10a1c8c35 [term 1 LEADER]: Becoming Leader. State: Replica: 7881b29d47df4d188edfd8d10a1c8c35, State: Running, Role: LEADER
I20260807 18:20:42.224689 18776 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7881b29d47df4d188edfd8d10a1c8c35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7881b29d47df4d188edfd8d10a1c8c35" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40565 } } peers { permanent_uuid: "2eb638ca160c415aa129ed6fed171acd" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35929 } } peers { permanent_uuid: "3a1be66c26f6486b8969d0da789d9503" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34417 } }
I20260807 18:20:42.230652 18785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7881b29d47df4d188edfd8d10a1c8c35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7881b29d47df4d188edfd8d10a1c8c35. Latest consensus state: current_term: 1 leader_uuid: "7881b29d47df4d188edfd8d10a1c8c35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7881b29d47df4d188edfd8d10a1c8c35" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40565 } } peers { permanent_uuid: "2eb638ca160c415aa129ed6fed171acd" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35929 } } peers { permanent_uuid: "3a1be66c26f6486b8969d0da789d9503" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34417 } } }
I20260807 18:20:42.231395 18785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7881b29d47df4d188edfd8d10a1c8c35 [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:42.239722 18794 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:20:42.263590 18794 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260807 18:20:42.280969 18563 tablet.cc:2369] T 00000000000000000000000000000000 P 2eb638ca160c415aa129ed6fed171acd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:20:42.296978 18740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:42.255378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38304 (local address 127.17.110.61:35929)
0807 18:20:42.255638 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:42.255649 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:42.263738 (+ 8089us) server_negotiation.cc:407] Connection header received
0807 18:20:42.264191 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:42.264202 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:42.264336 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:42.264444 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:42.278899 (+ 14455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:42.279754 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:42.282100 (+ 2346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:42.282980 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:42.283142 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:42.287210 (+ 4068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:42.287219 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:42.287224 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:42.287305 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:42.292118 (+ 4813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:42.293132 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:42.293138 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:42.293142 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:42.293364 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:42.293881 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:42.293888 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:42.293892 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:42.294419 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:42.294790 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:42.296142 (+ 1352us) server_negotiation.cc:299] Negotiation successful
0807 18:20:42.296513 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 18:20:42.302307 18692 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3a1be66c26f6486b8969d0da789d9503 [term 1 FOLLOWER]: Refusing update from remote peer 7881b29d47df4d188edfd8d10a1c8c35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:20:42.304158 18785 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7881b29d47df4d188edfd8d10a1c8c35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a1be66c26f6486b8969d0da789d9503" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:20:42.323222 18592 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2eb638ca160c415aa129ed6fed171acd [term 1 FOLLOWER]: Refusing update from remote peer 7881b29d47df4d188edfd8d10a1c8c35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:20:42.323697 18739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:42.255175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48200 (local address 127.17.110.62:40565)
0807 18:20:42.255417 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:42.255425 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:42.270428 (+ 15003us) server_negotiation.cc:407] Connection header received
0807 18:20:42.272174 (+ 1746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:42.272187 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:42.272315 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:42.272416 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:42.277870 (+ 5454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:42.279383 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:42.286590 (+ 7207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:42.287594 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:42.287729 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:42.295579 (+ 7850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:42.295589 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:42.295595 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:42.295657 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:42.301373 (+ 5716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:42.320726 (+ 19353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:42.320736 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:42.320740 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:42.320961 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:42.321529 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:42.321536 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:42.321540 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:42.321981 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:42.322256 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:42.322948 (+ 692us) server_negotiation.cc:299] Negotiation successful
0807 18:20:42.323288 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 18:20:42.324847 18785 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7881b29d47df4d188edfd8d10a1c8c35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2eb638ca160c415aa129ed6fed171acd" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:20:42.344645 18732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:42.271502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44138 (local address 127.17.110.60:34417)
0807 18:20:42.271751 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:42.271757 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:42.282764 (+ 11007us) server_negotiation.cc:407] Connection header received
0807 18:20:42.283361 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:42.283369 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:42.283473 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:42.283559 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:42.291741 (+ 8182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:42.293026 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:42.315954 (+ 22928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:42.316863 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:42.317005 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:42.326412 (+ 9407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:20:42.326421 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:20:42.326428 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:20:42.326485 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:20:42.330986 (+ 4501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:42.332284 (+ 1298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:42.332291 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:42.332295 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:42.332471 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:20:42.342241 (+ 9770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:20:42.342249 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:20:42.342254 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:20:42.342827 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:20:42.343144 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:42.343876 (+ 732us) server_negotiation.cc:299] Negotiation successful
0807 18:20:42.344203 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 18:20:42.346319 18743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a1be66c26f6486b8969d0da789d9503 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7881b29d47df4d188edfd8d10a1c8c35. Latest consensus state: current_term: 1 leader_uuid: "7881b29d47df4d188edfd8d10a1c8c35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7881b29d47df4d188edfd8d10a1c8c35" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40565 } } peers { permanent_uuid: "2eb638ca160c415aa129ed6fed171acd" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35929 } } peers { permanent_uuid: "3a1be66c26f6486b8969d0da789d9503" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34417 } } }
I20260807 18:20:42.346786 18743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a1be66c26f6486b8969d0da789d9503 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:20:42.347775 18819 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 18:20:42.357848 18785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7881b29d47df4d188edfd8d10a1c8c35 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7881b29d47df4d188edfd8d10a1c8c35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7881b29d47df4d188edfd8d10a1c8c35" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40565 } } peers { permanent_uuid: "2eb638ca160c415aa129ed6fed171acd" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35929 } } peers { permanent_uuid: "3a1be66c26f6486b8969d0da789d9503" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34417 } } }
I20260807 18:20:42.358196 18785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7881b29d47df4d188edfd8d10a1c8c35 [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:42.358726 18794 catalog_manager.cc:1347] Generated new cluster ID: e8add08816364565b29063d031e56933
I20260807 18:20:42.358847 18794 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:20:42.363318 18814 mvcc.cc:205] Tried to move back new op lower bound from 7315975546051764224 to 7315975545760845824. Current Snapshot: MvccSnapshot[applied={T|T < 7315975546051764224 or (T in {7315975546051764224})}]
I20260807 18:20:42.368434 18816 mvcc.cc:205] Tried to move back new op lower bound from 7315975546051764224 to 7315975545760845824. Current Snapshot: MvccSnapshot[applied={T|T < 7315975546051764224}]
I20260807 18:20:42.373356 18743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a1be66c26f6486b8969d0da789d9503 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7881b29d47df4d188edfd8d10a1c8c35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7881b29d47df4d188edfd8d10a1c8c35" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40565 } } peers { permanent_uuid: "2eb638ca160c415aa129ed6fed171acd" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35929 } } peers { permanent_uuid: "3a1be66c26f6486b8969d0da789d9503" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34417 } } }
I20260807 18:20:42.373701 18743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a1be66c26f6486b8969d0da789d9503 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:20:42.379374 18776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7881b29d47df4d188edfd8d10a1c8c35 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7881b29d47df4d188edfd8d10a1c8c35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7881b29d47df4d188edfd8d10a1c8c35" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40565 } } peers { permanent_uuid: "2eb638ca160c415aa129ed6fed171acd" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35929 } } peers { permanent_uuid: "3a1be66c26f6486b8969d0da789d9503" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34417 } } }
I20260807 18:20:42.379375 18760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2eb638ca160c415aa129ed6fed171acd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7881b29d47df4d188edfd8d10a1c8c35. Latest consensus state: current_term: 1 leader_uuid: "7881b29d47df4d188edfd8d10a1c8c35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7881b29d47df4d188edfd8d10a1c8c35" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40565 } } peers { permanent_uuid: "2eb638ca160c415aa129ed6fed171acd" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35929 } } peers { permanent_uuid: "3a1be66c26f6486b8969d0da789d9503" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34417 } } }
I20260807 18:20:42.379743 18760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2eb638ca160c415aa129ed6fed171acd [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:20:42.379755 18776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7881b29d47df4d188edfd8d10a1c8c35 [sys.catalog]: This master's current role is: LEADER
I20260807 18:20:42.397370 18794 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:20:42.399073 18760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2eb638ca160c415aa129ed6fed171acd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7881b29d47df4d188edfd8d10a1c8c35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7881b29d47df4d188edfd8d10a1c8c35" member_type: VOTER last_known_addr { host: "127.17.110.62" port: 40565 } } peers { permanent_uuid: "2eb638ca160c415aa129ed6fed171acd" member_type: VOTER last_known_addr { host: "127.17.110.61" port: 35929 } } peers { permanent_uuid: "3a1be66c26f6486b8969d0da789d9503" member_type: VOTER last_known_addr { host: "127.17.110.60" port: 34417 } } }
I20260807 18:20:42.399356 18760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2eb638ca160c415aa129ed6fed171acd [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:20:42.399622 18794 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:20:42.423062 18794 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7881b29d47df4d188edfd8d10a1c8c35: Generated new TSK 0
I20260807 18:20:42.424345 18794 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 18:20:42.599112 18739 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:20:42.583728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48208 (local address 127.17.110.62:40565)
0807 18:20:42.583986 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:42.583995 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:42.596514 (+ 12519us) server_negotiation.cc:407] Connection header received
0807 18:20:42.597154 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:42.597162 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:42.597267 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:42.597354 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:42.598817 (+ 1463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48208: BlockingRecv error: recv got EOF from 127.0.0.1:48208 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260807 18:20:42.601981 18740 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:20:42.583518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38312 (local address 127.17.110.61:35929)
0807 18:20:42.583788 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:42.583795 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:42.599970 (+ 16175us) server_negotiation.cc:407] Connection header received
0807 18:20:42.600284 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:42.600291 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:42.600410 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:42.600493 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:42.601658 (+ 1165us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38312: BlockingRecv error: recv got EOF from 127.0.0.1:38312 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
W20260807 18:20:42.602747 18732 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:20:42.600502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44154 (local address 127.17.110.60:34417)
0807 18:20:42.600704 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:42.600712 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:42.600827 (+ 115us) server_negotiation.cc:407] Connection header received
0807 18:20:42.601123 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:42.601129 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:42.601217 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:42.601309 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:42.602447 (+ 1138us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44154: BlockingRecv error: recv got EOF from 127.0.0.1:44154 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260807 18:20:42.653256 18740 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:20:42.639588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38324 (local address 127.17.110.61:35929)
0807 18:20:42.639823 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:42.639831 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:42.650750 (+ 10919us) server_negotiation.cc:407] Connection header received
0807 18:20:42.651445 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:42.651452 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:42.651598 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:42.651685 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:42.652940 (+ 1255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38324: BlockingRecv error: recv got EOF from 127.0.0.1:38324 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260807 18:20:42.709556 18740 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:20:42.687627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38336 (local address 127.17.110.61:35929)
0807 18:20:42.687820 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:42.687826 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:42.707494 (+ 19668us) server_negotiation.cc:407] Connection header received
0807 18:20:42.707869 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:42.707881 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:42.707996 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:42.708085 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:42.709283 (+ 1198us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38336: BlockingRecv error: recv got EOF from 127.0.0.1:38336 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260807 18:20:42.757318 18740 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:20:42.753118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38338 (local address 127.17.110.61:35929)
0807 18:20:42.753340 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:42.753347 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:42.755493 (+ 2146us) server_negotiation.cc:407] Connection header received
0807 18:20:42.755713 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:42.755719 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:42.755852 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:42.755998 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:42.756987 (+ 989us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38338: BlockingRecv error: recv got EOF from 127.0.0.1:38338 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260807 18:20:42.771673 18740 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:20:42.768874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38346 (local address 127.17.110.61:35929)
0807 18:20:42.769196 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:42.769202 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:42.769857 (+ 655us) server_negotiation.cc:407] Connection header received
0807 18:20:42.770087 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:42.770093 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:42.770207 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:42.770346 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:42.771418 (+ 1072us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38346: BlockingRecv error: recv got EOF from 127.0.0.1:38346 (error 108)
Metrics: {"server-negotiator.queue_time_us":131}
W20260807 18:20:42.786911 18740 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:20:42.784027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38358 (local address 127.17.110.61:35929)
0807 18:20:42.784234 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:42.784247 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:42.784878 (+ 631us) server_negotiation.cc:407] Connection header received
0807 18:20:42.785232 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:42.785239 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:42.785386 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:42.785541 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:42.786528 (+ 987us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38358: BlockingRecv error: recv got EOF from 127.0.0.1:38358 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260807 18:20:42.809546 18740 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:20:42.807009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38360 (local address 127.17.110.61:35929)
0807 18:20:42.807209 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:42.807216 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:42.807842 (+ 626us) server_negotiation.cc:407] Connection header received
0807 18:20:42.808055 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:42.808062 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:42.808171 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:42.808302 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:42.809272 (+ 970us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38360: BlockingRecv error: recv got EOF from 127.0.0.1:38360 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260807 18:20:42.848209 18740 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:20:42.845698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38372 (local address 127.17.110.61:35929)
0807 18:20:42.845891 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:42.845896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:42.846364 (+ 468us) server_negotiation.cc:407] Connection header received
0807 18:20:42.846599 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:42.846603 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:42.846755 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:42.846862 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:42.847926 (+ 1064us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38372: BlockingRecv error: recv got EOF from 127.0.0.1:38372 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260807 18:20:42.919829 18740 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:20:42.916848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38376 (local address 127.17.110.61:35929)
0807 18:20:42.917111 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:42.917122 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:42.917765 (+ 643us) server_negotiation.cc:407] Connection header received
0807 18:20:42.918075 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:42.918082 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:42.918213 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:42.918333 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:42.919470 (+ 1137us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38376: BlockingRecv error: recv got EOF from 127.0.0.1:38376 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 18:20:43.039409 18756 catalog_manager.cc:1259] Loaded cluster ID: e8add08816364565b29063d031e56933
I20260807 18:20:43.039547 18756 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3a1be66c26f6486b8969d0da789d9503: loading cluster ID for follower catalog manager: success
I20260807 18:20:43.042434 18756 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3a1be66c26f6486b8969d0da789d9503: acquiring CA information for follower catalog manager: success
I20260807 18:20:43.044788 18756 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3a1be66c26f6486b8969d0da789d9503: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 18:20:43.054857 18740 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:20:43.052759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38388 (local address 127.17.110.61:35929)
0807 18:20:43.052952 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:43.052957 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:43.053401 (+ 444us) server_negotiation.cc:407] Connection header received
0807 18:20:43.053576 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:43.053581 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:43.053665 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:43.053805 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:20:43.054539 (+ 734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38388: BlockingRecv error: recv got EOF from 127.0.0.1:38388 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 18:20:43.111390 18773 catalog_manager.cc:1259] Loaded cluster ID: e8add08816364565b29063d031e56933
I20260807 18:20:43.111511 18773 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2eb638ca160c415aa129ed6fed171acd: loading cluster ID for follower catalog manager: success
I20260807 18:20:43.114053 18773 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2eb638ca160c415aa129ed6fed171acd: acquiring CA information for follower catalog manager: success
I20260807 18:20:43.116019 18773 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2eb638ca160c415aa129ed6fed171acd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:20:43.323275 18740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:43.316932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38392 (local address 127.17.110.61:35929)
0807 18:20:43.317130 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:43.317135 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:43.317410 (+ 275us) server_negotiation.cc:407] Connection header received
0807 18:20:43.317646 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:43.317651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:43.317830 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:43.317950 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:20:43.318394 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:43.319206 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:43.320721 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:43.321164 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:43.321283 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:43.321885 (+ 602us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:20:43.322474 (+ 589us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:20:43.322629 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:43.322912 (+ 283us) server_negotiation.cc:299] Negotiation successful
0807 18:20:43.323010 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:20:43.337750 18919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:43.330602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44156 (local address 127.17.110.60:34417)
0807 18:20:43.331425 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:43.331430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:43.331449 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:20:43.331570 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:43.331574 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:43.331753 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:43.331917 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:20:43.332438 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:43.333331 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:43.335044 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:43.335653 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:43.335770 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:43.336427 (+ 657us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:20:43.337124 (+ 697us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:20:43.337253 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:43.337392 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 18:20:43.337475 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":709,"thread_start_us":584,"threads_started":1}
I20260807 18:20:43.353190 18740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:43.344961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38402 (local address 127.17.110.61:35929)
0807 18:20:43.345162 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:43.345169 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:43.346187 (+ 1018us) server_negotiation.cc:407] Connection header received
0807 18:20:43.346502 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:43.346508 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:43.346613 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:43.346683 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:20:43.347651 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:43.348846 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:43.350745 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:43.351211 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:43.351398 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:43.352038 (+ 640us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:20:43.352219 (+ 181us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:20:43.352307 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:43.352558 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 18:20:43.352685 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 18:20:43.353884 18919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:43.346481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44160 (local address 127.17.110.60:34417)
0807 18:20:43.346678 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:43.346684 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:43.347364 (+ 680us) server_negotiation.cc:407] Connection header received
0807 18:20:43.347584 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:43.347589 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:43.347693 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:43.347783 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:20:43.348398 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:43.349309 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:43.351599 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:43.352145 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:43.352296 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:43.353084 (+ 788us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:20:43.353203 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:20:43.353310 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:43.353458 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 18:20:43.353548 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 18:20:43.354698 18925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:20:43.344830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48220 (local address 127.17.110.62:40565)
0807 18:20:43.345502 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0807 18:20:43.345511 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:20:43.345989 (+ 478us) server_negotiation.cc:407] Connection header received
0807 18:20:43.346345 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:20:43.346356 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:20:43.346659 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:20:43.346905 (+ 246us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:20:43.347865 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:43.349093 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:43.351555 (+ 2462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:20:43.352171 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:20:43.352330 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:20:43.353146 (+ 816us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:20:43.353674 (+ 528us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:20:43.353798 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 18:20:43.354279 (+ 481us) server_negotiation.cc:299] Negotiation successful
0807 18:20:43.354385 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":372,"threads_started":1}
I20260807 18:20:43.364744 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 18424
I20260807 18:20:43.366104 18515 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:20:43.481381 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 18424
I20260807 18:20:43.509904 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 18531
I20260807 18:20:43.511350 18615 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:20:43.623915 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 18531
I20260807 18:20:43.654418 17848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu pid 18636
I20260807 18:20:43.655809 18721 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:20:43.768831 17848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDgKS9Z/build/asan/bin/kudu with pid 18636
2026-08-07T18:20:43Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5462 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (11024 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (414139 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST